Skip to content

Enable to filter searched origins by visit types

Antoine Lambert requested to merge generated-differential-D5064-source into generated-differential-D5064-target

That diff adds support to filter origins by visit types in swh-search.

The following changes have been made:

  • Add a new visit_types field to elasticsearch document for origin.

  • Add a new optional visit_types parameter to origin_search method in SearchInterface.

  • Implement visit types filtering in search backends.

  • Send origin visit types to elasticsearch when processing origin visits in journal client.

I have tested that I could populate a local elasticsearch instance with those new data using the following command.

$ swh --log-level DEBUG search -C ~/.config/swh/search.yml journal-client objects -o origin_visit

I used the following configuration file:

search:
  cls: elasticsearch
  hosts:
    - http://localhost:9200

journal:
  brokers:
    - kafka1.internal.softwareheritage.org
    - kafka2.internal.softwareheritage.org
    - kafka3.internal.softwareheritage.org
    - kafka4.internal.softwareheritage.org
    
  prefix: swh.journal.objects
  group_id: anlambert.search

Related to swh-web#2869 (closed)

Migrated from D5064 (view on Phabricator)

Merge request reports