- May 24, 2022
-
- May 20, 2022
- May 17, 2022
-
-
vlorentz authored
-
- May 16, 2022
-
-
vlorentz authored
base_snapshot_reverse_branches needs to contain all objects that may be a snapshot target that the remote did not send to us. Because we now use all snapshots to build the "have" list, such targets include all targets of snapshots of "parent" origins, not just the previous snapshot of the current origin. This typically happens when a forge-fork pull branches from its parent. Resolves Sentry issue [[ https://sentry.softwareheritage.org/share/issue/1c08f5d764e7494e83ba254dc47f17af/ | SWH-LOADER-GIT-102 ]]
-
vlorentz authored
Only instances of (subclasses of) AbstractHttpGitClient have this attribute. For other instances, we can consider it to be False, because the dumb protocol only exists over HTTP(S). This issue was found by mypy, thanks to the addition of type annotations to changes in dulwich 0.20.36 affecting mypy's type inference.
-
- May 13, 2022
-
-
vlorentz authored
Before this commit, determine_wants() used the origin's last snapshot if any, or the closest parent's snapshot if not. However, we noticed that many repositories that are very slow to load are forks that were already visited, but their owner rebased it on the parent since the last visit, causing potentially many commits to be added to the origin. This ensures we do not needlessly fetch these new commits when we already loaded the parent.
-
- May 06, 2022
-
- May 02, 2022
-
-
Pratyush authored
-
- Apr 27, 2022
-
-
Antoine Lambert authored
Recent changes in swh-scheduler add new parameters to the celery tasks produced from swh.scheduler.model.ListedOrigin instances. So ensure to handle any new parameters by not hardcoding the expected ones in task signatures. Rename date parameter to visit_date in from disk loader tasks and make it non mandatory. Add new tests checking task parameters produced from ListedOrigin instances do no raise error when attempting to create a git loader. Related to T4187
- Apr 26, 2022
-
-
vlorentz authored
-
- Apr 21, 2022
-
-
Antoine Lambert authored
That hook can be frustrating as it can discard a long commit message if it finds a typo in it so better removing it.
-
vlorentz authored
-
- Apr 20, 2022
-
-
vlorentz authored
-
- Apr 08, 2022
-
-
Antoine Lambert authored
-
Antoine Lambert authored
Related to T3922
-
Antoine Lambert authored
black is considered stable since release 22.1.0 and the version we are currently using is quite outdated and not compatible with click 8.1.0, so it is time to bump it to its latest stable release. Please note that E501 pycodestyle warning related to line length is replaced by B950 one from flake8-bugbear as recommended by black. https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length Related to T3922
-
- Mar 22, 2022
-
-
Antoine Lambert authored
Due to test modules being copied in subdirectories of the build directory by setuptools, it makes pytest fail by raising ImportPathMismatchError exceptions when invoked from root directory of the module. So ignore the build folder to discover tests.
-
- Feb 10, 2022
-
-
Antoine Lambert authored
To install the new hook: $ pre-commit install -t commit-msg
-
- Jan 21, 2022
-
-
Antoine R. Dumont authored
This currently fails the origin visit and update the visit status to 'failed'. This got listed by listers but current access to such origin is actually private, it'd probably make sense to make the status of the visit as not_found instead. This takes care of the most frequent issue so (460k) [1]. [1] https://sentry.softwareheritage.org/share/issue/3a3663f8cc424a48999af28728152ef0/
-
- Jan 14, 2022
-
-
vlorentz authored
swh-model 5.0.0 removes these arguments from the constructor.
-
vlorentz authored
This allows representing git trees with disordered entries, as the "normal" data model requires them to be sorted.
-
vlorentz authored
This allows representing all git objects instead of rejecting objects that do not fit in our "normal" data model. This commit is restricted to revisions and releases for now, a future commit will add directories.
-
- Jan 11, 2022
-
-
Antoine Lambert authored
urljoin does not produce the same output if the base URL does not have a trailing slash. >>> from urllib.parse import urljoin >>> urljoin("https://git.example.org/repo", "info/refs") 'https://git.example.org/info/refs' >>> urljoin("https://git.example.org/repo/", "info/refs") 'https://git.example.org/repo/info/refs' So ensure the base URL ends with a slash to avoid generating invalid URLs and make loading failed.
-
Nicolas Dandrimont authored
-
- Jan 10, 2022
-
-
vlorentz authored
instead of writing them all at once, which partially defeats the point of using a spooled buffer.
-
vlorentz authored
'requests' does the job just fine with less complexity.
-
vlorentz authored
response.content_type is set by Dulwich, but isn't part of urllib3's HTTPResponse, so we shouldn't rely on it. (And it makes mypy complain when the 'types-urllib3' package is installed)
-
- Dec 20, 2021
-
-
vlorentz authored
This mock was clunky because it didn't actually behave much like dulwich's Tag. Additionally, a future commit will need to access the as_raw_chunks() method of ShaFile objects, so SWHTag isn't suitable anymore as it would need to diverge even more by implementing its own serialization.
-
- Dec 16, 2021
-
-
Antoine R. Dumont authored
This also drops spurious copyright headers to those files if present. Related to T3812
-
- Oct 28, 2021
-
-
Antoine R. Dumont authored
This: - unifies this parameter name with names similar to what's used in lister - also documents it better Related to T3695
-
- Oct 21, 2021
-
- Oct 20, 2021
-
-
Antoine R. Dumont authored
-
- Oct 11, 2021
-
- Oct 05, 2021
-
-
Antoine Lambert authored
Some dumb git servers might reference a no longer existing pack file while it is possible to fully load a repository without it. So remove bogus pack file from the global packs list when encountering such edge case and try to continue the loading anyway. Related to T3618
-
Antoine Lambert authored
An error was raised previously when trying to fetch HEAD. Related to T3618
-
- Oct 01, 2021
-
-
Antoine R. Dumont authored
-