- Feb 16, 2017
-
-
Nicolas Dandrimont authored
-
- Feb 14, 2017
-
-
Nicolas Dandrimont authored
To make sure corruptions such as T680 don't happen again, use the same normalization function as swh.model before inserting timestamps into our database. This makes swh.storage reject non-integer timestamp values as well. Update tests to reflect this change.
-
- Feb 09, 2017
-
-
Nicolas Dandrimont authored
Close T672
-
Nicolas Dandrimont authored
-
Antoine Pietri authored
-
Antoine Pietri authored
-
- Feb 07, 2017
-
-
Nicolas Dandrimont authored
The buckets are using the last two bytes of the object id, so that we spread the load across different lines on sequential archivings.
-
- Jan 26, 2017
-
-
Antoine R. Dumont authored
Closes T646
-
- Jan 03, 2017
-
-
Nicolas Dandrimont authored
-
- Dec 31, 2016
-
-
Antoine R. Dumont authored
-
- Dec 20, 2016
-
-
Antoine R. Dumont authored
-
- Nov 15, 2016
-
-
Antoine R. Dumont authored
-
- Nov 03, 2016
-
-
Nicolas Dandrimont authored
The check_config method allows a dynamic check of the configuration for a running storage. We can make sure that we have proper permissions on the object storage as well as the database before running things.
-
- Oct 13, 2016
-
-
Antoine R. Dumont authored
Related T575
-
- Sep 29, 2016
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
- Sep 23, 2016
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
The 'unknown sha1 path' cannot happen in the default archiver since it reads from the archive db (so the fallback code is not necessary in the worker). To the contrary, since 'archiver to backend' reads from stdin (for now), we could have unregistered sha1s from that source. This commit makes the director deal with that before sending sha1 to workers. It's also the director's job to set the state to 'missing' when the force_copy is true before sending sha1 to worker.
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
- Sep 22, 2016
-
-
Antoine R. Dumont authored
Before that, it could for example pushed copies to other mirrors where the content was missing.
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Closes T569
-
- Sep 21, 2016
-
-
Antoine R. Dumont authored
-
- Sep 20, 2016
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
If you want the archiver to have cloud abilities, install the package python3-swh.objstorage.cloud on the server as well.
-
- Sep 17, 2016
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Logging an entry about it
-
Antoine R. Dumont authored
- Only read the storage key once. - Improve the logging error.
-
- Sep 16, 2016
-
-
Antoine R. Dumont authored
Also, adds a force_copy flag in the configuration file to avoid checking preexistence of sha1. This is to be efficient for the first time copy in a new backend.
-
Antoine R. Dumont authored
The initial director and worker had split configuration files. Now it's unified to be both archiver/worker.yml file
-
- Sep 15, 2016
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-