Skip to content
Snippets Groups Projects

Use a standard postgresql db fixture in test_cli

Closed Use a standard postgresql db fixture in test_cli
generated-differential-D7170-source into generated-differential-D7170-target
Closed David Douard requested to merge generated-differential-D7170-source into generated-differential-D7170-target

instead of a custom 'test_db' (which did not truncate dbversion and origin tables, not sure what the intent was for this).

Depends on !313 (closed)

Migrated from D7170 (view on Phabricator)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build has FAILED

    Patch application report for D7170 (id=25983)

    Could not rebase; Attempt merge onto 3139d334...

    Updating 3139d33..adeb54f
Fast-forward
 MANIFEST.in                                        |   2 +
 swh/core/cli/db.py                                 |  67 +++++++++++
 swh/core/db/db_utils.py                            | 130 +++++++++++++++++++--
 swh/core/db/sql/35-dbmetadata.sql                  |  28 +++++
 swh/core/db/tests/conftest.py                      |  40 +++++++
 .../tests/data/cli/{1-schema.sql => 30-schema.sql} |   2 +-
 .../db/tests/data/cli/{3-func.sql => 40-funcs.sql} |   0
 .../db/tests/data/cli/{4-data.sql => 50-data.sql}  |   0
 swh/core/db/tests/test_cli.py                      |  71 ++++-------
 swh/core/db/tests/test_db_utils.py                 |  78 +++++++++++++
 10 files changed, 363 insertions(+), 55 deletions(-)
 create mode 100644 swh/core/db/sql/35-dbmetadata.sql
 rename swh/core/db/tests/data/cli/{1-schema.sql => 30-schema.sql} (87%)
 rename swh/core/db/tests/data/cli/{3-func.sql => 40-funcs.sql} (100%)
 rename swh/core/db/tests/data/cli/{4-data.sql => 50-data.sql} (100%)
 create mode 100644 swh/core/db/tests/test_db_utils.py
    Changes applied before test
    commit adeb54f770209284f0924a7e4d5bd5977a567527
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 17:54:01 2022 +0100

    Use a standard postgresql db fixture in test_cli
    
    instead of a custom 'test_db' (which did not truncate dbversion and
    origin tables, not sure what the intent was for this).

commit d50695c3feb57b1e21d93e78364a943232fe91ff
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 15:02:51 2022 +0100

    Add support for dbversion and dbmodule handling in swh db init
    
    for now, managing the dbversion table (storing the db schema upgrade
    history) is partially the responsibility of the swh modules implementing
    a datastore (swh-storage etc.), and the actual swh module was not stored
    in the database.
    
    This adds support for both the dbversion and dbmodule tables management
    in swh.core.db.
    
    The `swh db init` command now creates both these tables and initialize
    their values if possible.
    For this, it introduces a new common "API" for swh.core.db based datastores:
    
    - a swh module implementing a datastore is expected to have a
      `get_datastore` function in its top namespace; this function is
      typically the actual `get_storage`, `get_scheduler` etc. functions
    - this factory function is expected to use the "postgresql" cls for the
      datastore based on swh.core.db,
    - the datastore instance (eg. instance of the swh.storage.postgresql.Storage
      class) is expected to have a `get_current_version()` method (returning
      an int).
    
    This revision also provides a new `swh db version` command displaying
    the current (code) version (if available), the db version (possibly the whole
    version history) and flavor (if any) for the datastore given as argument.
    
    It should be backward compatible with existing swh datastore modules (i.e.
    not yet adapted to this new API).

commit b1e449fd91ebcc403966a79fd62d449921fcf550
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 16:00:43 2022 +0100

    Refactor the mock_package_sql fixture to allow multiple sql script set
    
    will be used to implement several test scenarios in following commits.
    Also rename existing sql scripts with a "legit" numbering scheme.

    Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/315/ See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/315/console

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D7170 (id=25983)

    Could not rebase; Attempt merge onto 3139d334...

    Updating 3139d33..adeb54f
Fast-forward
 MANIFEST.in                                        |   2 +
 swh/core/cli/db.py                                 |  67 +++++++++++
 swh/core/db/db_utils.py                            | 130 +++++++++++++++++++--
 swh/core/db/sql/35-dbmetadata.sql                  |  28 +++++
 swh/core/db/tests/conftest.py                      |  40 +++++++
 .../tests/data/cli/{1-schema.sql => 30-schema.sql} |   2 +-
 .../db/tests/data/cli/{3-func.sql => 40-funcs.sql} |   0
 .../db/tests/data/cli/{4-data.sql => 50-data.sql}  |   0
 swh/core/db/tests/test_cli.py                      |  71 ++++-------
 swh/core/db/tests/test_db_utils.py                 |  78 +++++++++++++
 10 files changed, 363 insertions(+), 55 deletions(-)
 create mode 100644 swh/core/db/sql/35-dbmetadata.sql
 rename swh/core/db/tests/data/cli/{1-schema.sql => 30-schema.sql} (87%)
 rename swh/core/db/tests/data/cli/{3-func.sql => 40-funcs.sql} (100%)
 rename swh/core/db/tests/data/cli/{4-data.sql => 50-data.sql} (100%)
 create mode 100644 swh/core/db/tests/test_db_utils.py
    Changes applied before test
    commit adeb54f770209284f0924a7e4d5bd5977a567527
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 17:54:01 2022 +0100

    Use a standard postgresql db fixture in test_cli
    
    instead of a custom 'test_db' (which did not truncate dbversion and
    origin tables, not sure what the intent was for this).

commit d50695c3feb57b1e21d93e78364a943232fe91ff
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 15:02:51 2022 +0100

    Add support for dbversion and dbmodule handling in swh db init
    
    for now, managing the dbversion table (storing the db schema upgrade
    history) is partially the responsibility of the swh modules implementing
    a datastore (swh-storage etc.), and the actual swh module was not stored
    in the database.
    
    This adds support for both the dbversion and dbmodule tables management
    in swh.core.db.
    
    The `swh db init` command now creates both these tables and initialize
    their values if possible.
    For this, it introduces a new common "API" for swh.core.db based datastores:
    
    - a swh module implementing a datastore is expected to have a
      `get_datastore` function in its top namespace; this function is
      typically the actual `get_storage`, `get_scheduler` etc. functions
    - this factory function is expected to use the "postgresql" cls for the
      datastore based on swh.core.db,
    - the datastore instance (eg. instance of the swh.storage.postgresql.Storage
      class) is expected to have a `get_current_version()` method (returning
      an int).
    
    This revision also provides a new `swh db version` command displaying
    the current (code) version (if available), the db version (possibly the whole
    version history) and flavor (if any) for the datastore given as argument.
    
    It should be backward compatible with existing swh datastore modules (i.e.
    not yet adapted to this new API).

commit b1e449fd91ebcc403966a79fd62d449921fcf550
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 16:00:43 2022 +0100

    Refactor the mock_package_sql fixture to allow multiple sql script set
    
    will be used to implement several test scenarios in following commits.
    Also rename existing sql scripts with a "legit" numbering scheme.

    See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/320/ for more details.

  • Antoine R. Dumont mentioned in merge request !316 (closed)

    mentioned in merge request !316 (closed)

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Maintainer

    Merge request was accepted

  • David Douard
    David Douard @douardda ·
    Author Maintainer

    rebase

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D7170 (id=26050)

    Could not rebase; Attempt merge onto 3139d334...

    Updating 3139d33..f8a07df
Fast-forward
 MANIFEST.in                                        |   2 +
 swh/core/cli/db.py                                 |  69 ++++++++++-
 swh/core/db/db_utils.py                            | 132 +++++++++++++++++++--
 swh/core/db/sql/35-dbmetadata.sql                  |  28 +++++
 swh/core/db/tests/conftest.py                      |  45 +++++++
 .../tests/data/cli/{1-schema.sql => 30-schema.sql} |   2 +-
 .../db/tests/data/cli/{3-func.sql => 40-funcs.sql} |   0
 .../db/tests/data/cli/{4-data.sql => 50-data.sql}  |   0
 swh/core/db/tests/test_cli.py                      |  73 ++++--------
 swh/core/db/tests/test_db_utils.py                 |  83 +++++++++++++
 10 files changed, 376 insertions(+), 58 deletions(-)
 create mode 100644 swh/core/db/sql/35-dbmetadata.sql
 rename swh/core/db/tests/data/cli/{1-schema.sql => 30-schema.sql} (87%)
 rename swh/core/db/tests/data/cli/{3-func.sql => 40-funcs.sql} (100%)
 rename swh/core/db/tests/data/cli/{4-data.sql => 50-data.sql} (100%)
 create mode 100644 swh/core/db/tests/test_db_utils.py
    Changes applied before test
    commit f8a07dfa0bf668905d66ad88b25b2e1f3f569177
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 17:54:01 2022 +0100

    Use a standard postgresql db fixture in test_cli
    
    instead of a custom 'test_db' (which did not truncate dbversion and
    origin tables, not sure what the intent was for this).

commit c4bd270c3000df52176dafbf5c7f16e283e918c8
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 15:02:51 2022 +0100

    Add support for dbversion and dbmodule handling in swh db init
    
    for now, managing the dbversion table (storing the db schema upgrade
    history) is partially the responsibility of the swh modules implementing
    a datastore (swh-storage etc.), and the actual swh module was not stored
    in the database.
    
    This adds support for both the dbversion and dbmodule tables management
    in swh.core.db.
    
    The `swh db init` command now creates both these tables and initialize
    their values if possible.
    For this, it introduces a new common "API" for swh.core.db based datastores:
    
    - a swh module implementing a datastore is expected to have a
      `get_datastore` function in its top namespace; this function is
      typically the actual `get_storage`, `get_scheduler` etc. functions
    - this factory function is expected to use the "postgresql" cls for the
      datastore based on swh.core.db,
    - the datastore instance (eg. instance of the swh.storage.postgresql.Storage
      class) is expected to have a `get_current_version()` method (returning
      an int).
    
    This revision also provides a new `swh db version` command displaying
    the current (code) version (if available), the db version (possibly the whole
    version history) and flavor (if any) for the datastore given as argument.
    
    It should be backward compatible with existing swh datastore modules (i.e.
    not yet adapted to this new API).

commit b1e449fd91ebcc403966a79fd62d449921fcf550
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 28 16:00:43 2022 +0100

    Refactor the mock_package_sql fixture to allow multiple sql script set
    
    will be used to implement several test scenarios in following commits.
    Also rename existing sql scripts with a "legit" numbering scheme.

    See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/341/ for more details.

  • David Douard
    David Douard @douardda ·
    Author Maintainer

    Merge request was merged

  • David Douard closed

    closed

  • Phabricator Migration user approved this merge request

    approved this merge request

Please register or sign in to reply