Fix precision loss in Decimal parameter binding - #201
Open
aminghadersohi wants to merge 1 commit into
Open
aminghadersohi wants to merge 1 commit into
aminghadersohi wants to merge 1 commit into
Conversation
Signed-off-by: Amin Ghadersohi <amin.ghadersohi@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The ibm_db dialect declares
supports_native_decimal = False, so SQLAlchemy's Numeric bind processor converts PythonDecimalvalues to binary floats before sending them to the DBAPI. High-precision DECIMAL/NUMERIC parameters silently lose precision, including values used in predicates.Fix
Enable native decimal handling on
DB2Dialect_ibm_dbonly. No change to the shared base dialect or the ODBC/JDBC implementations, and no SQL rewriting or casts.This matches the DBAPI's behavior rather than emulating decimal support: ibm_db 3.2.3's Decimal binding path converts the Decimal to its exact string representation and binds it as
SQL_C_CHAR; ibm_db_dbi's result conversion produces Python Decimal results. The new raw-DBAPI live test passes before and after the dialect change, independently verifying this behavior for execute and executemany.Existing Numeric result conversion, including
asdecimal=False, is unchanged.Regression tests
test/test_numeric.pyfollows the repository's SQLAlchemy testing fixtures/assertions style. It covers DECIMAL and Numeric binding, wide positive/negative values, exponent notation, zero, NULL, ordinary integer/float inputs, result conversion, typed execute/executemany, exact Decimal-valued predicates, and direct DBAPI binding.Same test file, same Python/SQLAlchemy/DBAPI/server; changing only
ibm_db_sa/ibm_db.pybetween upstream master7f3866e3a827924055323fd5f10ac40b7a9fb2a7and this patch:The live failures are exact-comparison failures, not connection/setup failures:
123456789012345.67123456789012345.7000000000123456789012345.6700000000123456789012345678901.1234567890123456789012346000000.0000000000123456789012345678901.1234567890-123456789012345678901.1234567890-123456789012346000000.0000000000-123456789012345678901.1234567890No precision tolerance, rounding, decimal-context adjustment, or cast is used.
Reproduce connectionless tests:
For the live tests, supply a URL for a disposable database using the existing explicit
db2+ibm_dbentry point (the SQLAlchemy test plugin expands implicit driver names):python -m pytest -p sqlalchemy.testing.plugin.pytestplugin --dburi "$DB2_TEST_URL" -q test/test_numeric.pyValidation used Python 3.11.2, SQLAlchemy 2.0.52, ibm-db 3.2.3, pytest 9.1.1, and DB2 Community Edition 11.5.9.0. The SQLAlchemy pytest plugin emits unknown-marker warnings; no tests were skipped. Ruff 0.5.0 check/format for the new test file and
git diff --checkpass. The entire legacy upstream suite was not run.Additional live checks
Against the same local DB2 LUW 11.5.9.0 server, a separate SQLAlchemy Core check of connectivity, DDL, reflection, Unicode/quoted/NULL/repeated binds, typed round trips, commit/rollback visibility, dispose/reconnect and pool invalidation passed (8/8). Exact
DECIMAL(31,10)round trips (123456789012345678901.1234567890in and out, asDecimal) were confirmed through each existing URL alias:db2,db2+ibm_dbandibm_db_sa. Released 0.4.0 reproduces the decimal corruption with the same check.Not covered: other DBAPI or SQLAlchemy versions, DB2 editions/platforms other than LUW 11.5.9.0, ODBC/JDBC, TLS/external authentication. The ibm_db-based AS400 subclass inherits this flag but has not been tested against IBM i.