Use elasticsearch aliases to simplify maintenance operations
- Allow to explicitly configure the index and aliases names. It depreciates the prefix parameter.
- Use dedicated aliases for read and write operations
- manage them in the initialization method
Another diff will follow to implement the initialization during the server startup. (The tests of the server part need to be declared before)
Related to swh/infra/sysadm-environment#3076 (closed)
Migrated from D5179 (view on Phabricator)
Merge request reports
Activity
Build is green
Patch application report for D5179 (id=18518)
Rebasing onto 04dadef9...
Current branch diff-target is up to date.Changes applied before test
commit a8a9409c7a2ad60015239ad7955f187146cb3e37 Author: Vincent SELLIER <vincent.sellier@softwareheritage.org> Date: Mon Mar 1 22:08:54 2021 +0100 Use elasticsearch aliases to simplify maintenance operations - Use dedicated aliases for read and write operations - manage them in the initialization method Related to swh/infra/sysadm-environment#3076See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/92/ for more details.
-
Build is green
Patch application report for D5179 (id=18519)
Rebasing onto 04dadef9...
Current branch diff-target is up to date.Changes applied before test
commit 7053dca24f35eddca06d66f3a31687ce3f5e88f2 Author: Vincent SELLIER <vincent.sellier@softwareheritage.org> Date: Mon Mar 1 22:08:54 2021 +0100 Use elasticsearch aliases to simplify maintenance operations - Allow to explicitely configure the index and aliases names. It depreciates the prefix parameter. - Use dedicated aliases for read and write operations - manage them in the initialization method Related to swh/infra/sysadm-environment#3076See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/93/ for more details.
mentioned in merge request !42 (closed)
That assumes that
originis the only alias; and hopefully it won't be for long (#2073 (closed))Instead, it shouldn't be configurable on the CLI, and the config file should have an entry with a pair of names for each index
Will we used different indexes for #2073 (closed) ? Even with several indexes, It's not clear (for me at least) if using a unique read alias with several underlying indexes could be more advantageous. It will probably depend of how the search will be used from the api perspective. Perhaps it should be more prudent to keep this diff as simple as possible and implement the eventual improvements in #2073 (closed). WDYT?
-
! In !39 (closed), @vsellier wrote: Will we used different indexes for #2073 (closed) ?
It should allow searching objects that aren't origins (eg. directories), so yes
Even with several indexes, It's not clear (for me at least) if using a unique read alias with several underlying indexes could be more advantageous.
What does that mean? Can an alias reference multiple indexes? How does that work in terms of ids for example?
Perhaps it should be more prudent to keep this diff as simple as possible and implement the eventual improvements in #2073 (closed).
We could, but I don't see how it's more prudent; allowing a config like
{origin: {read: foo, write: bar}}isn't any harder than adding these CLI switches, so we might as well do it now What does that mean? Can an alias reference multiple indexes? How does that work in terms of ids for example? yes, an alias can reference multi indexes. If same ids are present in several indexes, the risk is to have duplicate result if the documents are matching the search.
We could, but I don't see how it's more prudent; allowing a config like
{origin: {read: foo, write: bar}}isn't any harder than adding these CLI switches, so we might as well do it now ok for such kind of configuration but just having read and write aliases is not enough as we need to know which mapping to apply on the index. Having a configuration like{origin: {index: name. read: foo, write: bar}}withorigina "static" indentifier should solve the problem.-
! In !39 (closed), @vsellier wrote: Having a configuration like
{origin: {index: name. read: foo, write: bar}}withorigina "static" indentifier should solve the problem.sure
Some references in the commit message have been migrated:
- T3076 is now swh/infra/sysadm-environment#3076 (closed)
-
Build is green
Patch application report for D5179 (id=18547)
Rebasing onto e9ffac4f...
Current branch diff-target is up to date.Changes applied before test
commit 7c795a603f7ac7ae43812991556c9ae574f476ce Author: Vincent SELLIER <vincent.sellier@softwareheritage.org> Date: Mon Mar 1 22:08:54 2021 +0100 Use elasticsearch aliases to simplify maintenance operations - Allow to explicitely configure the index and aliases names. It depreciates the prefix parameter. - Use dedicated aliases for read and write operations - manage them in the initialization method Related to swh/infra/sysadm-environment#3076See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/100/ for more details.
