Skip to content

Add Orchestrator

Tushar Goel requested to merge generated-differential-D5036-source into generated-differential-D5036-target

This is to build a mechanism to write to write the data from clearcode database which has been mapped with swh storage into swh RawExtrensicMetadata, and the data that has not been mapped to a table unmapped_data. This process of orchestration will run periodically and will only try to map new data that has been entered after the last orchestration process and the data that was not mapped in last orchestration.

Initialize tables if they don't exist in database. Initialize swh storage and add MetadataAuthority, MetadataFetcher, then map previously unmapped data and get last run date of orchestration then read data from clearcode and orchestor rows from clearcode DB (if whole row is mapped then in metadataStorage, if partial or no data is matched then store that row in unmapped data table for future mapping purpose), if tool of row is fossoloy then skip that row. Modify the function map_row (this will also return dicitionary of data from now) and make a new function to get type of tool of that row Add tests and docstrings.

Signed-off-by: Tushar Goel tushar.goel.dav@gmail.com

Migrated from D5036 (view on Phabricator)

Merge request reports