Skip to content

Add Luigi workflow in swh-dataset

It should:

  • export files to the local disk
  • upload to S3
  • define Athena tables

Migrated from T4676 (view on Phabricator)

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information