- Oct 13, 2023
-
-
VCS type now needs to be provided to swh.loader.core.nar.Nar constructor. See swh/devel/swh-loader-core@b6de012f.
-
- Oct 12, 2023
-
- Oct 10, 2023
-
-
vlorentz authored
-
- Jun 09, 2023
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
This also fixes the hg checkout related loader and task inconsistently named (which prevented them from being scheduled) Refs. swh/infra/sysadm-environment#4906
-
- Jun 05, 2023
-
-
Antoine R. Dumont authored
Otherwise, we'd lose the context in the snapshot. Refs. swh/meta#4979
-
Antoine R. Dumont authored
This also unifies with other loaders. Prior to this commit the visit was in status failed, now it's a not found like any other loaders. Refs. swh/meta#4979
-
- Jun 02, 2023
-
-
Antoine R. Dumont authored
This provides the class HgDirectoryLoader with only method `fetch_artifact`. It clones a repository at a specific changeset (or tag) and ingests the DAG objects from the resulting directory tree. It also checks the checksums (Nar or standard) if provided. Refs. swh/meta#4979
-
Antoine R. Dumont authored
-
- May 10, 2023
-
-
Antoine Lambert authored
When a mercurial origin is loaded, SWH stores the mapping between the original hg revision ids and SWH revision ids as extids. This mapping is then reused in subsequent loads to determine which new revisions should be archived. However it exist cases where previously seen hg revisions were removed from the mercurial repository between two loads by SWH. Previously the code filtering such no longer existing revisions was not catching the right exception and thus an error was raised when attempting to reload such mercurial origins. Related to #4583
-
- Feb 17, 2023
-
-
Antoine Lambert authored
Related to swh/meta#4960
-
- Feb 16, 2023
-
-
Jérémy Bobbio (Lunar) authored
Related to swh/meta#4959
-
- Feb 02, 2023
-
-
Antoine Lambert authored
This fixes python 3.7 support due to poetry, a dependency of isort, that removed support for that Python version in a recent release.
-
- Dec 19, 2022
-
-
Antoine Lambert authored
In order to remove warnings about /apidoc/*.rst files being included multiple times in toc when building full swh documentation, prefer to include module indices only when building standalone package documentation. Also include them the proper sphinx way. Related to T4496
-
- Oct 19, 2022
-
-
Antoine Lambert authored
-
- Oct 18, 2022
-
-
David Douard authored
- pre-commit from 4.1.0 to 4.3.0, - codespell from 2.2.1 to 2.2.2, - black from 22.3.0 to 22.10.0 and - flake8 from 4.0.1 to 5.0.4. Also freeze flake8 dependencies. Also change flake8's repo config to github (the gitlab mirror being outdated).
-
Antoine Lambert authored
Use helper fixture loading_task_creation_for_listed_origin_test from swh-loader-core and remove redundant tests.
-
- Sep 28, 2022
-
-
Stefan Sperling authored
Billiard version 4 is known to cause trouble in the git-loader's test suite because of specific requirements set by the celery package: pkg_resources.ContextualVersionConflict: (billiard 4.0.2 (/home/stsp/.virtualenvs/swh/lib/python3.9/site-packages), Requirement.parse('billiard<4.0,>=3.6.4.0'), {'celery'}) Until that is fixed, pin billiard to known-working versions.
-
- May 09, 2022
-
-
Pratyush authored
-
- Apr 27, 2022
-
-
Antoine Lambert authored
Recent changes in swh-scheduler add new parameters to the celery tasks produced from swh.scheduler.model.ListedOrigin instances. So ensure to handle any new parameters by not hardcoding the expected ones in task signatures. Remove unsafe use of unnamed task parameters. Add new tests checking task parameters produced from ListedOrigin instances do no raise error when attempting to create a hg loader. Related to T4187
-
- Apr 26, 2022
-
-
vlorentz authored
-
- Apr 21, 2022
-
-
Antoine Lambert authored
That hook can be frustrating as it can discard a long commit message if it finds a typo in it so better removing it.
-
vlorentz authored
-
- Apr 08, 2022
-
-
Antoine Lambert authored
-
Antoine Lambert authored
Related to T3922
-
Antoine Lambert authored
black is considered stable since release 22.1.0 and the version we are currently using is quite outdated and not compatible with click 8.1.0, so it is time to bump it to its latest stable release. Please note that E501 pycodestyle warning related to line length is replaced by B950 one from flake8-bugbear as recommended by black. https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length Related to T3922
-
- Apr 06, 2022
-
-
Antoine Lambert authored
pytest-postgresql 3.1.3 and pytest-redis 2.4.0 added support for pytest >= 7 so we can now drop the pytest pinning.
-
- Mar 22, 2022
-
-
Antoine Lambert authored
Due to test modules being copied in subdirectories of the build directory by setuptools, it makes pytest fail by raising ImportPathMismatchError exceptions when invoked from root directory of the module. So ignore the build folder to discover tests.
-
- Feb 10, 2022
-
-
Antoine Lambert authored
To install the new hook: $ pre-commit install -t commit-msg
-
- Feb 08, 2022
-
-
vlorentz authored
-
Raphaël Gomès authored
-
Raphaël Gomès authored
This used to always output `None`.
-
- Feb 07, 2022
-
-
Antoine R. Dumont authored
Related to T3916
-
- Jan 20, 2022
-
-
Raphaël Gomès authored
This "clone with timout" util just moved to core, let's use it. The tests are not relevant anymore, delete them.
-
- Dec 16, 2021
-
-
Antoine R. Dumont authored
This also drops spurious copyright headers to those files if present. Related to T3812
-
- Dec 01, 2021
-
-
Raphaël Gomès authored
See inline comments: in essence, currently the first revision in a run (incremental or not) will get the null revision as a parent and no cached directory, which creates the wrong diff for that revision (i.e. lots of files become added etc). We fix this problem by doing a full listing of the first revision in a run.
-
Raphaël Gomès authored
Filtering against the storage then asking for new revisions only makes sense if there are revisions not in storage and not ancestors of heads. If there are none, the current behavior lists all revisions, which is a whole lot of wasted work.
-
- Oct 11, 2021
-
-
Antoine R. Dumont authored
This is no longer required.
-
- Oct 02, 2021
-
-
Antoine R. Dumont authored
Related to T3612
-
- Sep 28, 2021
-
-
Antoine R. Dumont authored
Related to T3612
-