translator.py: Translate search query language to ES DSL
Translate swh search query language queries into Elasticsearch DSL
Depends on !64 (closed)
Migrated from D6025 (view on Phabricator)
Merge request reports
Activity
Build has FAILED
Patch application report for D6025 (id=21782)
Could not rebase; Attempt merge onto 2edbbbe8...
Updating 2edbbbe..757b046 Fast-forward query_language/grammar.js | 28 +++-- query_language/test/corpus/combinations.txt | 13 ++- swh/search/tests/test_translator.py | 111 +++++++++++++++++++ swh/search/translator.py | 161 ++++++++++++++++++++++++++++ 4 files changed, 300 insertions(+), 13 deletions(-) create mode 100644 swh/search/tests/test_translator.py create mode 100644 swh/search/translator.py
Changes applied before test
commit 757b0462d0bd11e8663c1e1d5f3de0b0ddc07773 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Summary: Translate swh search query language queries into Elasticsearch DSL Depends on !64 Test Plan: Reviewers: Subscribers: commit c27cb9a0a4d6a657e8ba02ab0f705bb77cc9b6ee Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 13 16:21:51 2021 +0530 grammar.js: Segregate sort_by and limit from filters Summary: The grammar should not allow using sort_by and limit more than once throughout the query. Unlike other filters, these two must not be concatenated by 'and' or 'or' Reviewers: #reviewers Differential Revision: https://forge.softwareheritage.org/!64
Link to build: https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/236/ See console output for more information: https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/236/console
Build has FAILED
Patch application report for D6025 (id=21787)
Rebasing onto 05efa541...
Current branch diff-target is up to date.
Changes applied before test
commit 715a2eb2206cd946bd3ad5fdc87b1cee3fb885ba Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 17:55:31 2021 +0530 translator.py: Import swh_ql.so using pkg_resources commit c3174265485ee9758507fd2fc41f52f46d35dd09 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:58:31 2021 +0530 translator.py: Complete all the filters and add new tests commit 3d030756b3f63082148253d44df06eb0ea1f447a Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
Link to build: https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/238/ See console output for more information: https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/238/console
Build has FAILED
Patch application report for D6025 (id=21803)
Rebasing onto 05efa541...
Current branch diff-target is up to date.
Changes applied before test
commit 16c254dc569b0d0cc8adb9ad1e6f70ad1ab39759 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 21:20:47 2021 +0530 translator: Try fixing swh_ql.so path commit 715a2eb2206cd946bd3ad5fdc87b1cee3fb885ba Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 17:55:31 2021 +0530 translator.py: Import swh_ql.so using pkg_resources commit c3174265485ee9758507fd2fc41f52f46d35dd09 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:58:31 2021 +0530 translator.py: Complete all the filters and add new tests commit 3d030756b3f63082148253d44df06eb0ea1f447a Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
Link to build: https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/239/ See console output for more information: https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/239/console
Build is green
Patch application report for D6025 (id=21805)
Rebasing onto 05efa541...
Current branch diff-target is up to date.
Changes applied before test
commit 905889ddf0f3f4676ed841c325774d3a6a72cef8 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 17:55:31 2021 +0530 translator.py: Fix resource path for tox environment commit c3174265485ee9758507fd2fc41f52f46d35dd09 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:58:31 2021 +0530 translator.py: Complete all the filters and add new tests commit 3d030756b3f63082148253d44df06eb0ea1f447a Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/240/ for more details.
Build is green
Patch application report for D6025 (id=21821)
Rebasing onto 05efa541...
Current branch diff-target is up to date.
Changes applied before test
commit 3575aac1c04744a934ef5b2ff5309b1fa084e03b Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Wed Jul 28 14:44:00 2021 +0530 translator: Changes suggested by vlorentz commit 905889ddf0f3f4676ed841c325774d3a6a72cef8 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 17:55:31 2021 +0530 translator.py: Fix resource path for tox environment commit c3174265485ee9758507fd2fc41f52f46d35dd09 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:58:31 2021 +0530 translator.py: Complete all the filters and add new tests commit 3d030756b3f63082148253d44df06eb0ea1f447a Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/241/ for more details.
Rebase:
- translator.py: Translate search query language to ES DSL
- translator.py: Complete all the filters and add new tests
- translator.py: Fix resource path for tox environment
- translator: Changes suggested by vlorentz
- translator.py: Update list of possible ql paths (suggested by vlorentz)
Build is green
Patch application report for D6025 (id=21857)
Rebasing onto c34011b7...
Current branch diff-target is up to date.
Changes applied before test
commit 3dab4f5d16d6f40e8ee4d1052211ab799183e3af Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Thu Jul 29 14:12:10 2021 +0530 translator.py: Update list of possible ql paths commit 847c5fa366445bab38096c2979ab74a11f876f06 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Wed Jul 28 14:44:00 2021 +0530 translator: Changes suggested by vlorentz commit c0b78c26c12c085062240b0d65969f04c89f9f6b Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 17:55:31 2021 +0530 translator.py: Fix resource path for tox environment commit 82a2dec905ecc66d1201687e17abd12312a5b038 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:58:31 2021 +0530 translator.py: Complete all the filters and add new tests commit 7d8feb2ab0373c4f05e6fbaf1371fe817fbc2a9f Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/243/ for more details.
- swh/search/translator.py 0 → 100644
272 } 273 } 274 return { 275 "range": { 276 f"{name}_date": { 277 self.RANGE_OPERATOR_MAP[op]: value.replace("Z", "+00:00"), 278 } 279 } 280 } 281 282 if category == "sortBy": 283 return value 284 285 if category == "limit": 286 return value 287 - swh/search/translator.py 0 → 100644
73 return {"bool": {"must": [filters1, filters2]}} 74 if conj_op == "or": 75 return {"bool": {"should": [filters1, filters2]}} 76 77 if node.type == "filter": 78 filter_category = node.children[0] 79 return self._parse_filter(filter_category) 80 81 if node.type == "sortBy": 82 return self._parse_filter(node) 83 84 if node.type == "limit": 85 return self._parse_filter(node) 86 87 return Exception( 88 f"Unknown node type ({node.type}) " Build is green
Patch application report for D6025 (id=21865)
Rebasing onto c34011b7...
Current branch diff-target is up to date.
Changes applied before test
commit f67cc169e66e1d30aaf41c3544b2d523480cc823 Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Fri Jul 30 00:19:21 2021 +0530 translator.py: Mention filter category and name in exception commit 2d596405d7571ef15ba0f19f7ed3dfd6b88ad1cb Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/244/ for more details.
Build is green
Patch application report for D6025 (id=21868)
Rebasing onto c34011b7...
Current branch diff-target is up to date.
Changes applied before test
commit 734370ade926508fa14d31595b27429794f2f52c Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/245/ for more details.
- swh/search/tests/test_translator.py 0 → 100644
152 "nested": { 153 "path": "intrinsic_metadata", 154 "query": { 155 "multi_match": { 156 "query": r"""word1 word2 \" ' word3""", 157 "fields": [ 158 get_expansion("keywords", ".") + "^2", 159 get_expansion("descriptions", "."), 160 ], 161 } 162 }, 163 } 164 } 165 } 166 167 _test_results(query, expected) - swh/search/translator.py 0 → 100644
13 "<": "lt", 14 ">=": "gte", 15 "<=": "lte", 16 } 17 18 def __init__(self): 19 ql_rel_paths = [ 20 "swh_ql.so", # installed 21 "../../query_language/swh_ql.so", # development 22 ] 23 for ql_rel_path in ql_rel_paths: 24 ql_path = resource_filename("swh.search", ql_rel_path) 25 if os.path.exists(ql_path): 26 break 27 else: 28 assert False, "swh_ql.so was not found in any of the expected paths" Build is green
Patch application report for D6025 (id=21870)
Rebasing onto c34011b7...
Current branch diff-target is up to date.
Changes applied before test
commit 3428e71a042b6e8b78de5c34c2fcc8cb1cae68ea Author: KShivendu <shivendu@iitbhilai.ac.in> Date: Tue Jul 27 00:18:11 2021 +0530 translator.py: Translate search query language to ES DSL Translate swh search query language queries into Elasticsearch DSL
See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/246/ for more details.