Commits · 6205fdee730bc35411974cf583d101d6eca84542 · Antoine Lambert / swh-dataset · GitLab

Snippets Groups Projects

Sep 23, 2020
- Adapt cli declaration entrypoint to swh.core 0.3 · 6205fdee
  David Douard authored 4 years ago
  
  v0.1.0
  
  6205fdee
Sep 18, 2020
- graph: replace deprecated persistent_identifier method · d2ebe07e
  Thibault Allançon authored 4 years ago
  
  Use the new SWHID naming convention instead of SWH PID.
  d2ebe07e
Sep 17, 2020
- python: Reorder imports with isort · 48ba2a40
  Antoine Lambert authored 4 years ago
  
  Related to T2610
  48ba2a40
- pre-commit: Add isort hook and configuration · c33e8ea7
  Antoine Lambert authored 4 years ago
  
  Related to T2610
  c33e8ea7
- pre-commit: Update flake8 hook configuration · d17281dd
  Antoine Lambert authored 4 years ago
  
  flake8 hook has been removed from https://github.com/pre-commit/pre-commit-hooks so now use the one from https://gitlab.com/pycqa/flake8
  d17281dd
Sep 10, 2020
- cli: speedup the `swh` cli command startup time · f71b2958
  David Douard authored 4 years ago
  
  by moving import statements in functions. Related to T2575.
  f71b2958
Jun 08, 2020
- graph export: compute node/edge type stats · 8d79b847
  Antoine Pietri authored 4 years ago
  
  8d79b847
May 15, 2020
- exporter: increase message.max.bytes value to allow large messages · d26a4247
  Antoine Pietri authored 4 years ago
  
  d26a4247
May 14, 2020
- graph exporter: fix process arguments, run process on content · 312474e9
  Antoine Pietri authored 4 years ago
  
  312474e9
- exporter: send EOF token to progress queue when process() finishes normally · 51897339
  Antoine Pietri authored 4 years ago
  
  51897339
- graph exporter: more exhaustive heuristic to remove pull requests · b608f414
  Antoine Pietri authored 4 years ago
  
  b608f414
May 05, 2020
- graph: do not deduplicate different visits from the same origin · 56fee877
  Antoine Pietri authored 4 years ago
  
  56fee877
- graph: use an sqlite3 on-disk set to avoid processing nodes twice · 7a34fb11
  Antoine Pietri authored 4 years ago
  
  7a34fb11
May 04, 2020
- Add Black pre-commit · 1796e402
  Antoine Pietri authored 4 years ago
  
  1796e402
- graph: update journal.fixer -> storage.fixer · 3f7771b9
  Antoine Pietri authored 4 years ago
  
  3f7771b9
- setup.py: reenable cli entrypoint · 563a061b
  Antoine Pietri authored 4 years ago
  
  563a061b
- exporter: minor style changes · 4da11489
  Antoine Pietri authored 4 years ago
  
  4da11489
- tests/graph: check the presence of duplicate nodes · b90e64c7
  Antoine Pietri authored 4 years ago
  
  b90e64c7
- Graph export: add unit tests · f88b89cb
  Antoine Pietri authored 4 years ago
  
  f88b89cb
- Rework graph export pipeline · 7088a960
  Antoine Pietri authored 4 years ago
  
  7088a960
- Add docstrings and types · daafb683
  Antoine Pietri authored 4 years ago
  
  daafb683
- dataset: add graph export based on kafka · d0c09c78
  Antoine Pietri authored 5 years ago
  
  d0c09c78
Apr 29, 2020
- setup.py: add documentation link · f874a911
  Stefano Zacchiroli authored 4 years ago
  
  f874a911
Apr 15, 2020
- setup.py: temporarily remove entrypoint until diff is merged · be9e71ba
  Antoine Pietri authored 4 years ago
  
  be9e71ba
Feb 05, 2020
- Bootstrap Python project files · 889d6636
  Antoine Pietri authored 5 years ago
  
  v0.0.1
  
  889d6636
Jan 20, 2020
- docs: Fix sphinx warnings · 085f2765
  Antoine Lambert authored 5 years ago
  
  Related to T2188
  085f2765
Nov 21, 2019
- Fix a few typos reported by pre-commit hooks · 3fad1a09
  David Douard authored 5 years ago
  
  3fad1a09
- Add a pre-commit config file · 38dee6c7
  David Douard authored 5 years ago
  
  38dee6c7
Nov 11, 2019
- docs/graph: postgresql: use latest/ url fragment instead of hardcoded date · 146208d4
  Antoine Pietri authored 5 years ago
  
  146208d4
Oct 01, 2019
- typing: minimal changes to make a no-op mypy run pass · 6be40304
  Stefano Zacchiroli authored 5 years ago
  
  6be40304
- init.py: switch to the canonical way of extending path · 4d3da901
  Stefano Zacchiroli authored 5 years ago
  
  4d3da901
- README: remove boilerplate in favor of minimal description · b57ee171
  Stefano Zacchiroli authored 5 years ago
  
  b57ee171
Sep 15, 2019
- make swh.dataset a proper (yet empty) package · 5b7d1880
  Stefano Zacchiroli authored 5 years ago
  
  5b7d1880
Sep 03, 2019
- docs: add missing TOC · 2ee3b7a3
  Antoine Pietri authored 5 years ago
  
  2ee3b7a3
Sep 01, 2019
- graph: athena: add CLI options for teasers · 3fcc3b6f
  Antoine Pietri authored 5 years ago
  
  3fcc3b6f
Aug 30, 2019
- schema: finish writing · f930c89d
  Antoine Pietri authored 5 years ago
  
  f930c89d
- WIP schema · 9d1f9c1d
  Antoine Pietri authored 5 years ago
  
  9d1f9c1d
- Add an Azure Databricks tutorial · bfd5288a
  Antoine Pietri authored 5 years ago
  
  bfd5288a
- Add an introduction sentence for teaser datasets · 98482478
  Antoine Pietri authored 5 years ago
  
  98482478
- Add missing graph/ · 7b9decde
  Antoine Pietri authored 5 years ago
  
  7b9decde