migrating to psycopg3
-
swh-storage swh-storage!1160 (merged) -
swh-deposit swh-deposit!452 (merged) -
swh-icinga-plugins swh-icinga-plugins!64 (closed) -
swh-indexer swh-indexer!537 (merged) -
swh-lister swh-lister!542 (merged) -
swh-loader-core swh-loader-core!538 (closed) -
swh-objstorage swh-objstorage!206 (merged) -
swh-scheduler swh-scheduler!400 (merged) -
swh-scrubber swh-scrubber!77 (merged) -
swh-vault swh-vault!210 (merged) -
swh-web swh-web!1362 (merged)
And these is a few remaining check to do :
-
do have a "dropping password from connection info" issue.
Merge request reports
Activity
Jenkins job DCORE/gitlab-builds #191 succeeded .
See Console Output and Coverage Report for more details.Jenkins job DCORE/gitlab-builds #192 succeeded .
See Console Output and Coverage Report for more details.added 45 commits
-
224ece90...2b8ce87b - 44 commits from branch
swh/devel:master
- e8e0edc8 - psycopg3: test pass…
-
224ece90...2b8ce87b - 44 commits from branch
Jenkins job DCORE/gitlab-builds #281 succeeded in 1 min 12 sec.
See Console Output, Blue Ocean and Coverage Report for more details.added 3 commits
-
abd147e2...6368526c - 2 commits from branch
swh/devel:master
- 46c3e31f - Migrate to psycopg3
-
abd147e2...6368526c - 2 commits from branch
Jenkins job DCORE/gitlab-builds #286 succeeded in 1 min 9 sec.
See Console Output, Blue Ocean and Coverage Report for more details.Jenkins job DCORE/gitlab-builds #287 succeeded in 1 min 5 sec.
See Console Output, Blue Ocean and Coverage Report for more details.mentioned in merge request swh-storage!1160 (merged)
124 126 @staticmethod 125 def adapt_conn(conn: psycopg2.extensions.connection): 126 """Makes psycopg2 use 'bytes' to decode bytea instead of 127 def adapt_conn(conn: psycopg.Connection[Any]): 128 """Makes psycopg use 'bytes' to decode bytea instead of 127 129 'memoryview', for this connection.""" 128 t_bytes = psycopg2.extensions.new_type((17,), "bytea", typecast_bytea) 129 psycopg2.extensions.register_type(t_bytes, conn) 130 # XXX-psycophg3: deal with this 131 # t_bytes = psycopg.extensions.new_type((17,), "bytea", typecast_bytea) 132 # psycopg.extensions.register_type(t_bytes, conn) 130 133 131 t_bytes_array = psycopg2.extensions.new_array_type((1001,), "bytea[]", t_bytes) 132 psycopg2.extensions.register_type(t_bytes_array, conn) 134 # t_bytes_array = psycopg.extensions.new_array_type((1001,), "bytea[]", t_bytes) 135 # psycopg.extensions.register_type(t_bytes_array, conn) changed this line in version 11 of the diff
62 62 ls: list, of, strings 63 63 li: 1, 2, 3, 4 64 64 """ 65 conffile.open("w").write(conf_contents) 65 with open(conffile, "w") as f: 66 f.write(conf_contents) changed this line in version 7 of the diff
@vlorentz I think this is the last unresolved comment from you on this MR.
Should we delete this code or are we missing a test?
From psycopg3 documentation:
Python types representing binary objects (bytes, bytearray, memoryview) are converted by default to bytea fields. By default data received is returned as bytes.
So if I understand things correctly, this is already properly taken care of an I'll drop this code.
- Resolved by Pierre-Yves David
28 28 cursor.execute("SHOW %s" % option) 29 29 old_value = cursor.fetchall()[0][0] 30 30 if old_value != value: 31 cursor.execute("SET LOCAL %s TO %%s" % option, (value,)) 31 # XXX-psycopg3: use SQL oriented formatting instead 32 cursor.execute( 33 "SELECT set_config(%s::text, %s::text, true)", (option, value) changed this line in version 6 of the diff
623 573 624 574 def parse_dsn_or_dbname(dsn_or_dbname: str) -> Dict[str, str]: 625 """Parse a psycopg2 dsn, falling back to supporting plain database names as well""" 575 """Parse a psycopg dsn, falling back to supporting plain database names as well""" 626 576 try: 627 return _parse_dsn(dsn_or_dbname) 628 except psycopg2.ProgrammingError: 629 # psycopg2 failed to parse the DSN; it's probably a database name, 577 d = conninfo_to_dict(dsn_or_dbname) 578 except psycopg.ProgrammingError: 579 # psycopg failed to parse the DSN; it's probably a database name, 630 580 # handle it as such 631 return _parse_dsn(f"dbname={dsn_or_dbname}") 581 d = conninfo_to_dict(f"dbname={dsn_or_dbname}") 582 # typing note: we don't pass **kwargs to conninfo_to_dict, so we don't get 583 # "int/None" as value and can safely cast to `Dict[str, str]` - Comment on lines +582 to +583
changed this line in version 7 of the diff
1 1 hypothesis >= 3.11.0 2 psycopg_pool changed this line in version 7 of the diff