- Aug 29, 2019
-
-
Antoine R. Dumont authored
-
- Aug 28, 2019
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Example use case: swh lister run --lister gitlab \ --priority high \ --policy oneshot \ --db-url postgresql://postgres@localhost:5432/swh-listers \ api_baseurl=https://gitlab.ow2.org/api/v4/ Related T1919
-
Antoine R. Dumont authored
Prior to this commit, the policy and priority were hard-coded. The default values are now the old hard-coded values. This will allow to develop a cli to trigger forges listing with oneshot policy and some priority tasks. Thus ingesting those faster and without manual interventation as we currently do.
-
- Jul 19, 2019
-
-
Archit Agrawal authored
Implement a packagist lister to list the names and metadata url of all the packages. Closes 1776
-
- Jul 18, 2019
-
-
Archit Agrawal authored
Add tests for pypi lister Closes T1890
-
Archit Agrawal authored
There were previously no tests for the listers which are using the class SimpleLister(like pypi) Refractored test_lister.py of lister core to accomodate tests for SimpleLister keeping the tests undisturbed for other lister.
-
- Jul 11, 2019
-
-
Stefano Zacchiroli authored
-
- Jul 04, 2019
-
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
- Jun 28, 2019
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Archit Agrawal authored
Remove the need to visit every page and extract the origin url by introducing a parameter url_prefix. The origin url is in format <prefix>/<repo_name> where The prefix is same for all the repos for a particular cgit instance.
-
Archit Agrawal authored
Some cgit instance have a pagination. Modifiy lister to find all the pages and list all the repos from all the pages.
-
Archit Agrawal authored
Implemented a lister to list the repos for a given CGit instance. Closes T1659
-
Antoine Lambert authored
Related T1659
-
- Jun 27, 2019
-
-
Archit Agrawal authored
Add description in task_dict method because the only metadata that can be found for a package at CRAN is its decsription. That can only br achived from the build in API in R, which ister is already using. Hence instead of getting metadata in loader, it is passed by lister.
-
- Jun 26, 2019
-
-
Antoine R. Dumont authored
Related D1635
-
Antoine R. Dumont authored
instead of converting that column as a string As a side effect, bitbucket wise, we provided improperly the after query parameter as a date not url encoded. This resulted in improper api response from bitbucket's (we received from time to time the same next index as the current one). Related T1826
- Jun 25, 2019
-
-
Antoine R. Dumont authored
If nothing has been done prior to a full relisting, there is actually nothing to list. So the relister in question does nothing. In that context, the IndexingLister class's `db_partition_indices` method now returns an empty list instead of raising a ValueError when there is nothing to list. Related T1826 Related e129e48c
-
Antoine R. Dumont authored
Prior to this commit, indexing lister instances were flushing every packet of 20. This can now be defined per sub classes.
-
Antoine R. Dumont authored
- Remove unneeded return instructions - Clarify tests code regarding request_index computations
-
Antoine R. Dumont authored
-
- Jun 21, 2019
-
-
Antoine R. Dumont authored
This is independent but still, it somehow fixes the issue occurring on T1826. Related T1826
-
Antoine R. Dumont authored
Related T1826
-
- Jun 20, 2019
-
-
Antoine R. Dumont authored
It's redundant with the swh modules in itself.
-
Archit Agrawal authored
Add new page in lister tutorial which guides through the process of running a new lister in docker.
-
Archit Agrawal authored
Add testing section for lister. Also added how to run a new lister section which elaborates about the steps required to run the now lister in docker
-
- Jun 19, 2019
-
-
Archit Agrawal authored
Change origin type from 'gnu' to 'tar'
-
vlorentz authored
They are useless.
-
Nancy Chauhan authored
-
- Jun 18, 2019
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Related T1824 Related P438
-
Antoine R. Dumont authored
As phabricator is an "instance" lister (there exists multiple instances of phabricator in the wild), we need to reference that information. In effect, this aligns phabricator lister with for example the gitlab one. Related T1801 Related P434
-
Antoine R. Dumont authored
Related P434
-