Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Register
  • Sign in
  • P puppet-swh-site
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 4
    • Issues 4
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 3
    • Merge requests 3
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Platform
  • Infrastructure
  • Puppet
  • puppet-swh-site
  • Issues
  • #579

Puppetize subset of indexers to start indexing computations

We can already start the indexing computations (language, mimetype) of the already copied 6M contents stored in azure. We don't have to wait for to have the ad-hoc implementation (T575) which triggers the indexing computations once the data is copied to azure (via the archiver-azure)

This task is to leverage #327 (closed)'s work to ease deploying worker. This concerns 3 task queues workers: orchestrator (T576), mimetype (T577), language (T578).

For now:

  • swh-storage is packaged ready (will need to upgrade the db to install new endpoint functions)
  • swh-indexer is packaged ready as well.
  • a priori, puppet manifests locally ready but i'm unsure of some stuff (so not pushed) - i need to test first (concat private/public config for one, factorizing by inheriting for another to avoid definition conflict...)

I intend to use worker01.azure worker for that test.


Migrated from T579 (view on Phabricator)

Assignee
Assign to
Time tracking