Commits · 7a6ea5da701044d578168f2aeadd387e03cbb61b · Platform / Development / snippets

May 12, 2022
- elastic-worker: Plug sentry setup as secret for loader git · 7a6ea5da
  Antoine R. Dumont authored 2 years ago
  
  Related to T4144
  7a6ea5da
May 11, 2022

elastic-worker: Deal with amqp credentials as secrets · 8dd6f1e2

This currently read the secrets and installs them properly in our config.yml the loader
uses to ingest origins.

It's currently convoluted but that's the simpler way for now. Short of changing the swh
tids and bits (loader, lister, ...) to actually allow using environment variables
instead of configuration here (sure but not right now).

Related to T4144

8dd6f1e2

Bump new release for elastic workers · 1e2c9088
Antoine R. Dumont authored 2 years ago
```
Related to T4144
```
1e2c9088

Apr 29, 2022
- elastic: Ignore internal files · 5222d5d0
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  5222d5d0
- elastic: Upgrade image to latest softwareheritage/loaders-2022-04-29 · e41dc37c
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  e41dc37c
- elastic: Update instructions · 9fe02ffb
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  9fe02ffb
- elastic: Ignore internal files · 3528d5c4
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  3528d5c4
- elastic: Decrease concurrency to 5 max · bcd4bbc9
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  bcd4bbc9
- elastic: Allow multiple loader deployments and update documentation · 105640ca
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  105640ca
- README: Update instructions · 8326c36e
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  8326c36e
- elastic: Use specific image loader version · 1190af7e
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  1190af7e
- elastic: Allow loader git to use credentials for metadata loading · 7df0359b
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  7df0359b
Apr 28, 2022
- worker: Align pipeline configuration with staging · c1475909
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  c1475909
- worker: Use softwareheritage/loaders image · 1bfec6e2
  Antoine R. Dumont authored 3 years ago
  
  Related to T4144
  1bfec6e2
Apr 08, 2022
- Add .git-blame-ignore-revs file with automatic reformatting commits · d557a58d
  Antoine Lambert authored 3 years ago
  
  d557a58d
Mar 15, 2022
- recheck_consistency: Remove useless code · fa9e387b
  vlorentz authored 3 years ago
  
  fa9e387b
Mar 14, 2022
- recheck_consistency: Waste less time dumping origins · f59e1115
  vlorentz authored 3 years ago
  
  f59e1115
- recheck_consistency: Add logging · b1650e2d
  vlorentz authored 3 years ago
  
  b1650e2d
Jan 31, 2022
- analyze_consistency_failures.py: Write fixed dirs · bdb48643
  vlorentz authored 3 years ago
  
  bdb48643
- analyze_consistency_failures.py: add missing writes · 402c6ed7
  vlorentz authored 3 years ago
  
  402c6ed7
- analyze_consistency_failures.py: copy fixes from check_consistency.py · 64ba0c64
  vlorentz authored 3 years ago
  
  so they are actually written to disk
  64ba0c64
- analyze_consistency_failures.py: recover dirs · b25f7331
  vlorentz authored 3 years ago
  
  b25f7331
Jan 27, 2022
- recheck_consistency.py: Reuse code from the backfiller. · e944cc63
  vlorentz authored 3 years ago
  
  e944cc63
- recheck_consistency.py: Better docstring · 7b95d164
  vlorentz authored 3 years ago
  
  7b95d164
- Add recheck_consistency.py · 0e0ca6ce
  vlorentz authored 3 years ago
  
  0e0ca6ce
Jan 26, 2022
- recover_corrupt_objects: clean imports · 04556110
  Nicolas Dandrimont authored 3 years ago
  
  04556110
- recover_corrupt_objects: use stream_results instead of stream_results_optional · 871da6ca
  Nicolas Dandrimont authored 3 years ago
  
  871da6ca
- recover_corrupt_objects: use logging instead of tqdm · 5e20eee1
  Nicolas Dandrimont authored 3 years ago
  
  5e20eee1
Jan 25, 2022
- recover_corrupt_objects: make assertions more useful · 99a3e499
  Nicolas Dandrimont authored 3 years ago
  
  - always print the swhid - only check author/committer fullname for revisions and releases - print a full dump of expected and returned revision/release objects
  99a3e499
- recover_corrupt_objects.py: Actually commit changes · cb2ced00
  vlorentz authored 3 years ago
  
  cb2ced00
- recover_corrupt_objects: Remove spurious select · 27369c56
  Nicolas Dandrimont authored 3 years ago
  
  27369c56
- recover_corrupt_objects: Pull config from a configfile instead of the cli · 44fe7877
  Nicolas Dandrimont authored 3 years ago
  
  44fe7877
- recover_corrupt_objects.py: Remove print, cleanup import · 3c8ad398
  vlorentz authored 3 years ago
  
  3c8ad398
Jan 24, 2022
- recover_corrupt_objects.py: Double-check insertion · 219ca5b6
  vlorentz authored 3 years ago
  
  219ca5b6
Jan 20, 2022
- Add recover_corrupt_objects.py · 7a7d0b30
  vlorentz authored 3 years ago
  
  7a7d0b30
Jan 17, 2022
- Add backup-weird-dirs-ok-in-kafka.py · 412f0c68
  vlorentz authored 3 years ago
  
  412f0c68
- analyze_consistency_failures: More heuristics · 39ccc5c6
  vlorentz authored 3 years ago
  
  39ccc5c6
- analyze_consistency_failures: Cache corrupt objects to save time · 6fef04f7
  vlorentz authored 3 years ago
  
  6fef04f7
Dec 01, 2021
- add swh-log-oneline.py; displays a one-line summary for each revision in history · 6a2068df
  Stefan Sperling authored 3 years ago
  
  6a2068df
- add swh-diff.py; displays SWH revision changes as a unified diff · b64f95c0
  Stefan Sperling authored 3 years ago
  
  b64f95c0