Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
S
swh-lister
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Antoine R. Dumont
swh-lister
Commits
09f3605a
Verified
Commit
09f3605a
authored
5 years ago
by
Antoine R. Dumont
Browse files
Options
Downloads
Patches
Plain Diff
docs: Remove spurious blank spaces
parent
4b2ab048
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
docs/run_a_new_lister.rst
+10
-10
10 additions, 10 deletions
docs/run_a_new_lister.rst
with
10 additions
and
10 deletions
docs/run_a_new_lister.rst
+
10
−
10
View file @
09f3605a
...
...
@@ -24,13 +24,13 @@ Here are the steps you need to follow to run a lister within your local environm
The file named `docker-compose.override.yml` will automatically be loaded by
``docker-compose``.Having an override makes it possible to run a docker container
with some swh packages installed from sources instead of using the latest
published packages from pypi. For more details, you may refer to README.md
with some swh packages installed from sources instead of using the latest
published packages from pypi. For more details, you may refer to README.md
present in ``swh-docker-dev``.
2. Follow the instruction mentioned under heading **Preparation steps** and
2. Follow the instruction mentioned under heading **Preparation steps** and
**Configuration file sample** in README.md of swh-lister.
3. Add in the lister configuration the new ``task_modules`` and ``task_queues``
entry for the your new lister. You need to amend the conf/lister.yml file to
entry for the your new lister. You need to amend the conf/lister.yml file to
add the entries. Here is an example for GNU lister::
celery:
...
...
@@ -45,7 +45,7 @@ Here are the steps you need to follow to run a lister within your local environm
4. Make sure to run ``storage (5002)`` and ``scheduler (5008)`` services locally.
You may use the following command to run docker::
~/swh-environment/swh-docker-dev$ docker-compose up -d
~/swh-environment/swh-docker-dev$ docker-compose up -d
5. Add the lister task-type in the scheduler. For example, if you want to
add pypi lister task-type ::
...
...
@@ -72,18 +72,18 @@ Here are the steps you need to follow to run a lister within your local environm
If your lister is creating new loading task not yet registered, you need
to register that task type as well.
6. Run your lister with the help of scheduler cli. You need to add the task in
6. Run your lister with the help of scheduler cli. You need to add the task in
the scheduler using its cli. For example, you need to execute this command
to run gnu lister ::
~/swh-environment$ swh scheduler --url http://localhost:5008/ task add \
list-gnu-full --policy oneshot
list-gnu-full --policy oneshot
After the execution of lister is complete, you can see the loading task created::
~/swh-environment/swh-lister$ swh scheduler task list
You can also check the repositories listed by the lister from the database in
You can also check the repositories listed by the lister from the database in
which the lister output is stored. To connect to the database::
~/swh-environment/swh-docker-dev$ docker-compose exec swh-lister bash -c \
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment