Some swh-search tests are failing since the release of elasticsearch 7.10

It looks like the new release of elasticsearch (7.10) introduced some breaking changes as some tests of swh-search are now failing.

For the record, I have executed the tests using the previous elasticsearch release (7.9.3) and they all passed.

---

Migrated from T2791 (view on Phabricator)

mentioned in merge request swh/infra/ci-cd/swh-jenkins-dockerfiles!22 (closed)

mentioned in merge request !28 (closed)

mentioned in merge request !120 (closed)

mentioned in merge request swh/infra/ci-cd/swh-jenkins-dockerfiles!45 (closed)

mentioned in merge request swh-environment!237 (closed)

added Archive search label

added priority:Normal label

assigned to @vlorentz

added priority:High label and removed priority:Normal label

I believe this to be a bug, ES ignores a specificed tokenizer (swh/meta$868, while earlier versions behaved like this: swh/meta$869).

I reported it here: https://discuss.elastic.co/t/es-7-10-ignores-analyzer-in-as-you-type-fields/256113

The upstream elasticsearch bug is fixed but not released yet. In the mean time, I think we can pin the version of ES in the jenkins docker image to one that doesn't have the bug (which I think @ardumont wants to do).

yes, i will try to do that and open a diff ;)

mentioned in commit swh/infra/ci-cd/swh-jenkins-dockerfiles@32b720b5

yes, i will try to do that and open a diff ;)

swh/infra/ci-cd/swh-jenkins-dockerfiles!45 (closed)

And as usual with this kind of diff, i'm unsure how to properly test it.

I just ensured through vagrant that i could indeed use the apt-get install -y elasticsearch=7.9.3 stanza and it worked, beats me how to do better though.

Build failed [1]

There is a check done at the end of the build which checks for upgrades and fails if there are some. In that case, there is an upgrade possible from 7.9.3 to 7.10.0 which we don't want (we do want 7.9.3). So the solution is to actually add a pin configuration version on 7.9.3 so version > 7.10.0 are ignored by apt.

Thanks to @olasd to suggest it to me ;)

I'll attend to that soon.

• [1] https://jenkins.softwareheritage.org/view/all/job/jenkins-tools/job/swh-jenkins-dockerfiles/lastBuild/consoleFull

mentioned in commit swh/infra/ci-cd/swh-jenkins-dockerfiles@6e742dbc

So swh-search build is back to green with ^ And the pile of diffs on search as well \m/

Given that the pinning is only excluding the 7.10.0, as soon as the 7.10.1 (assuming it's the release version with the fix), this should continue working with that version onwards.

So we should be able to close this.

I realized that dev wil have the problem locally as well (well most likely temporarily for such value of temporary...)

So I adapted the readme to explicit what needs to happen locally in !28 (closed).

Let's close this, everything is green (well blue).

I'm currently stuck on the debian packaging which i'll track in another task.

closed

mentioned in issue #2798 (closed)

mentioned in merge request !123 (closed)