Skip to content
Snippets Groups Projects

Unify loader instantiation

Closed Unify loader instantiation
generated-differential-D5071-source into generated-differential-D5071-target
Closed Antoine R. Dumont requested to merge generated-differential-D5071-source into generated-differential-D5071-target
  1. Feb 16, 2021
    • Antoine R. Dumont's avatar
      Unify loader instantiation · 7116bb75
      Antoine R. Dumont authored 
      This unifies and centralizes the instantiation the same way the lister does.

This introduces a new base class swh.loader.core.loader.Loader for all loaders whose
only concern for now is to instantiate loaders from either a configuration dict or a
configuration file.

This simplifies instantiation in celery task code and avoids duplicating the
configuration load in each loader constructor.

The end goal is to simplify the future refactoring on configuration. With the following,
we will only have to adapt the Loader class when we start simplifying uniformly the
configuration.

Also note that I mostly reused the equivalent `swh.lister.pattern.Lister.from_config*`.
I did not refactor the common behavior (to avoid throwing another dependency in the
mix). That could always be refactored later.

(inspired by both the work on listers and the configuration system work)

Related to T1410
      Verified
      7116bb75