Fix DB2 LUW reflection, binary binds and XML/DECFLOAT results - #205
Open
aminghadersohi wants to merge 1 commit into
Open
aminghadersohi wants to merge 1 commit into
aminghadersohi wants to merge 1 commit into
Conversation
Reflection on DB2 LUW (DB2Reflector): - get_foreign_keys filtered the constrained table by name only, so a same-named table in another schema contributed its keys, and it did not pair columns by key position. Read SYSCAT.REFERENCES with KEYCOLUSE, scoped by schema and name, ordered by COLSEQ; report ON DELETE/UPDATE rules. - get_pk_constraint and get_indexes split SYSCAT.INDEXES.COLNAMES on \w+, so a column such as "AMT$X" became two columns and DESC was lost; the primary key reported its backing index name. Read KEYCOLUSE/INDEXCOLUSE instead, report column_sorting, and skip DB2's internal XML region and path indexes. - get_unique_constraints now orders columns by key position. - get_check_constraints is implemented from SYSCAT.CHECKS. - has_sequence accepts the keywords SQLAlchemy 2's Inspector passes (info_cache), which raised TypeError. - get_columns reflects DECFLOAT, BINARY and VARBINARY (previously NullType) and CHAR/VARCHAR FOR BIT DATA as BINARY/VARBINARY, matching the bytes the DBAPI returns. ibm_db dialect: - Binary values are bound as bytes. dbapi.Binary is a memoryview, which ibm_db's executemany rejects for BINARY/VARBINARY/FOR BIT DATA (SQL0302N) and stores as its repr text in BLOB columns. - DECFLOAT results are returned as Decimal (the DBAPI returns str). - XML results drop the byte order mark and UTF-16 declaration the CLI adds when serializing. - is_disconnect recognizes a lost connection reported as the base ibm_db_dbi.Error (raised while fetching), and closing an already-closed connection no longer raises during pool cleanup. IBM i and z/OS reflectors are unchanged.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problems (found against a local DB2 LUW 11.5.9.0 server)
Reflection (
DB2Reflector, LUW only):get_foreign_keysfilters the constrained table by name only, so forA.CHILDit also returns the foreign keys ofB.CHILD. Columns of a composite key are not paired by key position.get_pk_constraintandget_indexessplitSYSCAT.INDEXES.COLNAMESon\w+, so a column"AMT$X"is reflected as two columnsamtandx.DESCindex columns lose their ordering. The primary key reports its backing index name (SQL2609...) instead of the constraint name.XMLcolumn is reflected as a user index.Inspector.has_sequenceraisesTypeError: has_sequence() got an unexpected keyword argument 'info_cache'.get_check_constraintsis not implemented.DECFLOAT,BINARYandVARBINARYreflect asNullTypewith a warning;CHAR/VARCHAR FOR BIT DATAreflect asCHAR/VARCHARalthough the DBAPI returnsbytes.ibm_dbdialect:executemanycorrupts data. SQLAlchemy binds binary values throughdbapi.Binary, whichibm_db_dbiimplements asmemoryview. Inexecutemany, ibm_db rejects amemoryviewforBINARY/VARBINARY/FOR BIT DATA(SQL0302N) and stores its repr text (<memory at 0x...>) inBLOBcolumns. Single-rowexecuteis fine. Reproduced with ibm_db 3.2.3 and 3.3.0. Bindingbytesworks in both paths.DECFLOATresults arestr; they are nowDecimal(orfloatwithasdecimal=False), andDecimalbinds are sent as exact text.<?xml version="1.0" encoding="UTF-16" ?>declaration synthesized by the CLI (even for a document stored with a UTF-8 declaration). DB2 stores XML without a declaration, so exactly that prefix is dropped.ibm_db_dbi.Error, whichis_disconnectignored, so the dead connection went back to the pool. After a server restart,close()on a dead connection raisesCLI0106E, and the pool logged an error for every connection it discarded; that one error is now ignored indo_close.IBM i (
AS400Reflector) and z/OS (OS390Reflector) are unchanged. The default-schema issue is handled separately in #203.Tests
New
test/test_luw_reflection.py, same style as the existing tests: connectionless processor/keyword checks and live reflection and round-trip checks (fixtures in two schemas with same-named tables, a composite FK with reversed key order, a"AMT$X"column, aDESCindex, an XML column, a check constraint, a sequence,DECFLOAT/BINARY/VARBINARY/FOR BIT DATA/BLOBvalues viaexecutemany).Same test file, Python, SQLAlchemy, DBAPI and server; only
ibm_db_sa/differs between upstream master7f3866eand this patch:test/test_luw_reflection.py(The unique-constraint test passes on master for this fixture; master already joins
TABCONST/KEYCOLUSEon schema and table.)python -m pytest -p sqlalchemy.testing.plugin.pytestplugin --dburi "db2+ibm_db://user:pass@host:port/db" -q test/test_luw_reflection.pyValidation: Python 3.11, SQLAlchemy 2.0.52, ibm-db 3.2.3, pytest 9.1.1, local throwaway DB2 Community Edition 11.5.9.0 container. The full legacy suite was not run.