core.db.cli: Add coverage and ensure `swh db *` works as expected

This explicitely checks the different scenario possible with the current subcommands (create, init-admin, init).

Using a subset of sql schema representation we have in our existing datasets.

Closes #2741 (closed)

Test Plan

tox

---

Migrated from D4381 (view on Phabricator)

Build is green

Patch application report for D4381 (id=15493)

Could not rebase; Attempt merge onto eefe1b63...

Updating eefe1b6..3abd57f
Fast-forward
 requirements-db.txt                             |   2 +-
 requirements-test-db.txt                        |   2 +-
 swh/core/cli/db.py                              |  54 +++++-
 swh/core/db/pytest_plugin.py                    |  10 +-
 swh/core/db/tests/data/cli/0-superuser-init.sql |   1 +
 swh/core/db/tests/data/cli/1-schema.sql         |  13 ++
 swh/core/db/tests/data/cli/3-func.sql           |   6 +
 swh/core/db/tests/data/cli/4-data.sql           |   5 +
 swh/core/db/tests/test_cli.py                   | 218 +++++++++++++++++++++++-
 9 files changed, 291 insertions(+), 20 deletions(-)
 create mode 100644 swh/core/db/tests/data/cli/0-superuser-init.sql
 create mode 100644 swh/core/db/tests/data/cli/1-schema.sql
 create mode 100644 swh/core/db/tests/data/cli/3-func.sql
 create mode 100644 swh/core/db/tests/data/cli/4-data.sql

Changes applied before test

commit 3abd57f7268c5eab1c470b6c3123f2fb1f26da24
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes #2741

commit 272f91b939af9846dbc9c26172b1ead36885e156
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 09:20:32 2020 +0100

    db.pytest_plugin: Make pytest-postgresql an optional runtime dependency
    
    Currently, this pulls the postgresql installation when installing swh.core
    which is not something wanted. That bypasses the current production pinning
    done in puppet and installs the latest postgresql-13 for example (even on node
    without postgresql).
    
    That particular dependency should be installed alongside swh modules which
    needs the core db pytest-plugin for their tests.

commit 745448399ebda14f08a9286b08fb6dd008d9504b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 28 18:27:46 2020 +0100

    cli.db: Open init-admin subcmd to initialize superuser-level scripts
    
    In staging, the db is already created by puppet, the current create database
    stanza forced within the create-db cannot work.
    
    Ths subsequent steps of postgresql extensions installation is required though.
    
    This will allow it.
    
    Related to swh/infra/sysadm-environment#2736

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/124/ for more details.

mentioned in merge request !311 (closed)

Thanks!

Merge request was accepted

approved this merge request

Rebase

Build is green

Patch application report for D4381 (id=15537)

Could not rebase; Attempt merge onto 12fa6f26...

Auto-merging swh/core/db/pytest_plugin.py
Merge made by the 'recursive' strategy.
 requirements-db.txt                             |   2 +-
 requirements-test-db.txt                        |   2 +-
 swh/core/db/pytest_plugin.py                    |  10 +-
 swh/core/db/tests/data/cli/0-superuser-init.sql |   1 +
 swh/core/db/tests/data/cli/1-schema.sql         |  13 ++
 swh/core/db/tests/data/cli/3-func.sql           |   6 +
 swh/core/db/tests/data/cli/4-data.sql           |   5 +
 swh/core/db/tests/test_cli.py                   | 212 +++++++++++++++++++++++-
 8 files changed, 240 insertions(+), 11 deletions(-)
 create mode 100644 swh/core/db/tests/data/cli/0-superuser-init.sql
 create mode 100644 swh/core/db/tests/data/cli/1-schema.sql
 create mode 100644 swh/core/db/tests/data/cli/3-func.sql
 create mode 100644 swh/core/db/tests/data/cli/4-data.sql

Changes applied before test

commit d4c5044c2bef39f4826bb6fb26c78b7e8b5f46f7
Merge: 12fa6f2 8438cfa
Author: Jenkins user <jenkins@localhost>
Date:   Fri Oct 30 14:55:28 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 8438cfa7ef0967d7c427d1fcc51816c1a1b5cab6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 09:20:32 2020 +0100

    db.pytest_plugin: Make pytest-postgresql an optional runtime dependency
    
    Currently, this pulls the postgresql installation when installing swh.core
    which is not something wanted. That bypasses the current production pinning
    done in puppet and installs the latest postgresql-13 for example (even on node
    without postgresql).
    
    That particular dependency should be installed alongside swh modules which
    needs the core db pytest-plugin for their tests.

commit e58c79a75316295d089312098119774034127f85
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes #2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/129/ for more details.

Rebase properly with the right commit

Build is green

Patch application report for D4381 (id=15538)

Rebasing onto 12fa6f26...

First, rewinding head to replay your work on top of it...
Applying: core.db.cli: Add coverage and ensure `swh db *` works as expected

Changes applied before test

commit adb34076e7061b167f99d59d73ae49487290e538
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes #2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/130/ for more details.

Rebase (master got updated)

Build is green

Patch application report for D4381 (id=15540)

Rebasing onto 12fa6f26...

Current branch diff-target is up to date.

Changes applied before test

commit 12b0e76defb15eebcc1fb2d79200842739e81fe6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes #2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/131/ for more details.

Some references in the commit message have been migrated:

• T2741 is now #2741 (closed)

Merge request was merged

closed