- Jun 12, 2019
- Jun 11, 2019
-
-
Archit Agrawal authored
Added a function which will derive the extension from filename and check if the fie extension match the type of file that is to be archived.
-
- Jun 08, 2019
-
-
Archit Agrawal authored
As discussed in T1389 to ingest all packages using base loader, it need a list of all the tarballs for a pakage. Hence modifified lister to recursively list all the tarballs for a package with their last updated time.
-
Archit Agrawal authored
Previously gnu lister was using same code as that of tarball loader to download, unzip and read tree.json file. To make the code consise the downloading method is changed to requests library.
-
Archit Agrawal authored
Implement first pass of gnu lister to list all the packages present in https://ftp.gnu.org/ Add GNU lister in README and cli.py Closes T1722
-
Archit Agrawal authored
Some of the new listers like GNU and CRAN do not follow the conventional way of making an HTTP request, hence they do not need some of the methods which are usually needed by in conventional HTTP request. But those method are marked abstractmethod in the core making them necessary to be present. So it is in best to remove abstractmethod to increase the readability of those listers.
-
- May 29, 2019
-
-
Antoine R. Dumont authored
This becomes a dictionary of key <lister-name>, value a dict of key <instance-name>, value list of dict username/password. Related T1772
- May 23, 2019
- May 22, 2019
-
-
David Douard authored
also add a cli group named 'lister' for the sake of consistency with other swh packages and rename the command as 'db-init', like: swh lister db-init LISTER [...]
-
- May 21, 2019
-
-
Antoine Lambert authored
Related T1508
-
- May 20, 2019
-
-
David Douard authored
This is (hopefully) a temporary fix that can be removed as soon as https://github.com/pypa/pip/issues/6239 is fixed, probably thanks to https://github.com/pypa/pip/issues/988
-
- May 15, 2019
-
-
Archit Agrawal authored
Add a lister of all hosted repositories on a Phabricator instance Closes T808
-
Antoine Lambert authored
-
- May 14, 2019
-
-
Antoine Lambert authored
-
- May 13, 2019
-
-
Antoine Lambert authored
-
- Apr 12, 2019
-
-
Nicolas Dandrimont authored
-
- Apr 11, 2019
-
-
David Douard authored
the exception can also be an AttributeError. Also do not reraise this exception (in github/tasks.py). This promise saving feature is used for tests.
- Apr 10, 2019
-
-
Antoine Lambert authored
Related T1508
-
Antoine Lambert authored
-
Antoine Lambert authored
Prior to this commit, initializing all listers was failing after the debian lister processing because of global insert_minimum_data init Related T1629
-
- Mar 24, 2019
-
-
Archit Agrawal authored
-
- Mar 19, 2019
-
-
Archit Agrawal authored
-
Archit Agrawal authored
-
Archit Agrawal authored
-
Archit Agrawal authored
-
Nicolas Dandrimont authored
Summary: Using order by and offset makes the partitioning a n^2 operation on the number of entries in the table, rather than an instant operation when using min/max. This assumes the indexable column is more or less uniform, which is not exactly true but not the worst approximation either. Test Plan: tox Reviewers: #reviewers, douardda Reviewed By: #reviewers, douardda Subscribers: douardda, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D1267
-
Nicolas Dandrimont authored
Summary: Using order by and offset makes the partitioning a n^2 operation on the number of entries in the table, rather than an instant operation when using min/max. This assumes the indexable column is more or less uniform, which is not exactly true but not the worst approximation either. Test Plan: tox Reviewers: #reviewers, douardda Reviewed By: #reviewers, douardda Subscribers: douardda, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D1267
-
Archit Agrawal authored
-
- Feb 14, 2019
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
- Feb 07, 2019
- Feb 06, 2019
-
-
David Douard authored
making 2 of them fail (in test_bb_lister.py and test_gh_lister.py).
-
David Douard authored
-
David Douard authored
-
David Douard authored
with conventional structure { 'cls': cls, 'args': {}} r
-
David Douard authored
-
David Douard authored
and add a 'all' possibel value for it, so that one can initialize all the database for all listers at once.
-