Skip to content
Snippets Groups Projects

indexer.storage: Change return types from Iterable to List

Closed indexer.storage: Change return types from Iterable to List
generated-differential-D4174-source into generated-differential-D4174-target
Closed vlorentz requested to merge generated-differential-D4174-source into generated-differential-D4174-target

For consistency with the main storage.

Depends on !263 (closed).

Migrated from D4174 (view on Phabricator)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build has FAILED

    Patch application report for D4174 (id=14685)

    Could not rebase; Attempt merge onto 51b10e89...

    Updating 51b10e8..be2c39e
Fast-forward
 swh/indexer/ctags.py                         |   2 +-
 swh/indexer/fossology_license.py             |  15 +-
 swh/indexer/indexer.py                       |  43 ++-
 swh/indexer/metadata.py                      |   6 +-
 swh/indexer/mimetype.py                      |  27 +-
 swh/indexer/origin_head.py                   |   4 +-
 swh/indexer/storage/__init__.py              | 187 +++++------
 swh/indexer/storage/api/client.py            |   3 +
 swh/indexer/storage/api/serializers.py       |  26 ++
 swh/indexer/storage/api/server.py            |   9 +-
 swh/indexer/storage/converters.py            |  15 +-
 swh/indexer/storage/db.py                    |  34 +-
 swh/indexer/storage/in_memory.py             |  98 +++---
 swh/indexer/storage/interface.py             |  57 ++--
 swh/indexer/tests/storage/conftest.py        |  10 +-
 swh/indexer/tests/storage/test_converters.py |  17 +-
 swh/indexer/tests/storage/test_metrics.py    |   8 +-
 swh/indexer/tests/storage/test_server.py     |  14 +-
 swh/indexer/tests/storage/test_storage.py    | 455 ++++++++++++++++-----------
 swh/indexer/tests/test_fossology_license.py  |  16 +
 swh/indexer/tests/test_mimetype.py           |   4 +-
 swh/indexer/tests/utils.py                   |  16 +-
 22 files changed, 615 insertions(+), 451 deletions(-)
 create mode 100644 swh/indexer/storage/api/serializers.py
    Changes applied before test
    commit be2c39ecedc25c5732378dde006e00de2de2f672
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 11:25:05 2020 +0200

    indexer.storage: Change return types from Iterable to List
    
    For consistency with the main storage.

commit 50428c652c86358cd20182ccc1735c519d6a0aee
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 11:06:15 2020 +0200

    license: use ContentLicenseRow in the storage interface instead of dicts.

commit 7fe4a89dbcf221a0125c474640cbcdc8b01b1df2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:54:35 2020 +0200

    base indexers: add type annotation for self.{storage,idx_storage}.

commit 486ee085f5ee999da38793815ae462c00ece4efb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:48:44 2020 +0200

    indexer.storage: Update docstrings of mimetype-related endpoints.

commit 4ec112337909cc364bec119b2ef6ae047a81b96f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:46:07 2020 +0200

    indexer.storage: Change return type annotation from Iterator to Iterable.
    
    When going through the RPC, it's turned into a list.

commit e8e94cf237471636e84ceac5d1740cfa7a982f90
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:01:26 2020 +0200

    tests: Enable type-checking on storage test functions.
    
    By adding a simple type annotation to the test functions' signature.

commit 44cee8f213f26e01aa63519a8c21962b2fca460b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 6 15:42:54 2020 +0200

    Make base indexers generic, with the result of index() as their type parameter.
    
    So the type of results can be statically checked, instead of needing to
    assert it to please mypy.

commit c3caf300830030f0cd1b3002d715e430582db8f1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 6 15:35:39 2020 +0200

    mimetype: use ContentMimetypeRow in the storage interface instead of dicts.
    
    This temporarily adds mess in the generic tests to support both rows and dicts,
    but I'll remove it once I migrated all endpoints.

    Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/75/ See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/75/console

  • vlorentz
    vlorentz @vlorentz ·
    Author Maintainer

    rebase

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D4174 (id=14689)

    Could not rebase; Attempt merge onto 51b10e89...

    Updating 51b10e8..b4d084d
Fast-forward
 swh/indexer/ctags.py                         |   4 +-
 swh/indexer/fossology_license.py             |  25 +-
 swh/indexer/indexer.py                       |  57 ++--
 swh/indexer/metadata.py                      |  38 ++-
 swh/indexer/mimetype.py                      |  31 +-
 swh/indexer/origin_head.py                   |  15 +-
 swh/indexer/storage/__init__.py              | 187 +++++------
 swh/indexer/storage/api/client.py            |   3 +
 swh/indexer/storage/api/serializers.py       |  26 ++
 swh/indexer/storage/api/server.py            |   9 +-
 swh/indexer/storage/converters.py            |  15 +-
 swh/indexer/storage/db.py                    |  34 +-
 swh/indexer/storage/in_memory.py             |  98 +++---
 swh/indexer/storage/interface.py             |  57 ++--
 swh/indexer/tests/storage/conftest.py        |  10 +-
 swh/indexer/tests/storage/test_converters.py |  17 +-
 swh/indexer/tests/storage/test_metrics.py    |   8 +-
 swh/indexer/tests/storage/test_server.py     |  14 +-
 swh/indexer/tests/storage/test_storage.py    | 455 ++++++++++++++++-----------
 swh/indexer/tests/test_ctags.py              |  25 +-
 swh/indexer/tests/test_fossology_license.py  |  22 +-
 swh/indexer/tests/test_indexer.py            |   4 +-
 swh/indexer/tests/test_mimetype.py           |  46 +--
 swh/indexer/tests/utils.py                   |  72 +----
 24 files changed, 701 insertions(+), 571 deletions(-)
 create mode 100644 swh/indexer/storage/api/serializers.py
    Changes applied before test
    commit b4d084d675bb89f5d6803ddbf75bbe54c3e91c03
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 11:25:05 2020 +0200

    indexer.storage: Change return types from Iterable to List
    
    For consistency with the main storage.

commit 570816fd87a1451e560fce8881a0da5626cf0ab4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 11:06:15 2020 +0200

    license: use ContentLicenseRow in the storage interface instead of dicts.

commit d346feffffaac7e8cc5c3bc0eedbffc66db1d37e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 11:43:05 2020 +0200

    all indexers: make index() return a list of results instead of a single one.
    
    1. it was wrongfully annotated as '-> TResult' even though some indexers
       can return None
    2. in a future commit, the fossology indexer will need to return multiple
       results.

commit 7fe4a89dbcf221a0125c474640cbcdc8b01b1df2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:54:35 2020 +0200

    base indexers: add type annotation for self.{storage,idx_storage}.

commit 486ee085f5ee999da38793815ae462c00ece4efb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:48:44 2020 +0200

    indexer.storage: Update docstrings of mimetype-related endpoints.

commit 4ec112337909cc364bec119b2ef6ae047a81b96f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:46:07 2020 +0200

    indexer.storage: Change return type annotation from Iterator to Iterable.
    
    When going through the RPC, it's turned into a list.

commit e8e94cf237471636e84ceac5d1740cfa7a982f90
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Oct 7 10:01:26 2020 +0200

    tests: Enable type-checking on storage test functions.
    
    By adding a simple type annotation to the test functions' signature.

commit 44cee8f213f26e01aa63519a8c21962b2fca460b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 6 15:42:54 2020 +0200

    Make base indexers generic, with the result of index() as their type parameter.
    
    So the type of results can be statically checked, instead of needing to
    assert it to please mypy.

commit c3caf300830030f0cd1b3002d715e430582db8f1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 6 15:35:39 2020 +0200

    mimetype: use ContentMimetypeRow in the storage interface instead of dicts.
    
    This temporarily adds mess in the generic tests to support both rows and dicts,
    but I'll remove it once I migrated all endpoints.

    See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/77/ for more details.

  • Antoine R. Dumont mentioned in merge request !265 (closed)

    mentioned in merge request !265 (closed)

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Maintainer

    Thanks.

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Maintainer

    Merge request was accepted

  • Antoine R. Dumont approved this merge request

    approved this merge request

  • vlorentz
    vlorentz @vlorentz ·
    Author Maintainer

    Merge request was merged

  • vlorentz closed

    closed

Please register or sign in to reply