- Jun 02, 2017
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Related T722
-
- Jun 01, 2017
-
-
Antoine R. Dumont authored
It's up to the indexer to query the storage to determine the indexer configuration's identifier and pass along the information when sending data for filter or update operations. Related T722
-
- May 29, 2017
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Related T722
-
- May 22, 2017
-
-
Antoine R. Dumont authored
-
- May 17, 2017
-
-
Antoine R. Dumont authored
Related T713
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
No rescheduling by default
-
Antoine R. Dumont authored
Prior to this, each indexer received the same batch of contents as the orchestrator. As some of our indexer need quite some time to finish when the batch size is huge enough, the only reasonable step was to reduce such batch size. But, in effect, this impacted as well other indexers which did not need such restriction (this for example creates more little db transations). The introduction of such option avoids that impact. This also introduces the option to check the presence on a per indexer basis. Related T713
-
- May 16, 2017
-
-
Nicolas Dandrimont authored
-
- May 12, 2017
-
-
Antoine R. Dumont authored
Default to no rescheduling.
-
Antoine R. Dumont authored
-
- May 04, 2017
-
-
Antoine R. Dumont authored
Related T712
-
- May 03, 2017
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
- May 02, 2017
-
-
Antoine R. Dumont authored
Related T713
-
Antoine R. Dumont authored
Related T713
-
Antoine R. Dumont authored
Related T712
-
- Apr 28, 2017
-
-
Antoine R. Dumont authored
Related T712
-
Antoine R. Dumont authored
Related T712
-
Antoine R. Dumont authored
If there are no hashes to recompute, do not read the content from objstorage. This optimization will only be useful for the case of the parameter recompute_checksum being false (which is the case in production for now) Related T712
-
- Apr 27, 2017
-
-
Antoine R. Dumont authored
Related T712
-
- Apr 26, 2017
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Related T712
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
-
- Mar 21, 2017
-
-
Antoine R. Dumont authored
Related T692
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Related T692 Closes D186
-
Antoine R. Dumont authored
The default behavior being to send directly hashes. This adaptation permits to send a dict whose key is specified as parameter of the script. Related T692
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
Related T692
-
Antoine R. Dumont authored
For all contents: - 'compute_checksums' option define checksums to (re)compute - check the content's already defined checksums against that list - If some are in 'compute_checksums' and already defined for that content, recompute those only if recompute_checksums is true. - Compute all other checksums mentioned in 'compute_checksums' for that content - update the content's metadata in db Related T692
-
Antoine R. Dumont authored
- Rename module from recompute to rehash - Rename options with more appropriate names - Improve methods' docstring and respect naming conventions - Fix wrong type value for option (use a boolean) - Fix wrong configuration key call Related T692
-
Antoine R. Dumont authored
This no longer fetches metadata from contents. The contents passed as parameter should be self-contained and in adequation with the 'primary_key' configuration option. As of now, that primary key is 'sha1'. Unknown or corrupted contents are skipped. Add some other improvments: - Improve docstrings - Align option names with _ as per usual conventions - Add a batch_size to retrieve contents' blobs - Add another batch_size for the update contents operations Related T692
-
Antoine R. Dumont authored
Related T692
-
Antoine R. Dumont authored
Related T692
-
- Mar 17, 2017
-
-
Antoine R. Dumont authored
Related T700
-