Skip to content

Fix precision loss in Decimal parameter binding - #201

Open
aminghadersohi wants to merge 1 commit into
ibmdb:masterfrom
aminghadersohi:fix-exact-decimal-binds
Open

aminghadersohi wants to merge 1 commit into
ibmdb:masterfrom
aminghadersohi:fix-exact-decimal-binds

Conversation

@aminghadersohi

@aminghadersohi aminghadersohi commented Sep 26, 2026 •

Copy link
Copy Markdown

Problem

The ibm_db dialect declares supports_native_decimal = False, so SQLAlchemy's Numeric bind processor converts Python Decimal values to binary floats before sending them to the DBAPI. High-precision DECIMAL/NUMERIC parameters silently lose precision, including values used in predicates.

Fix

Enable native decimal handling on DB2Dialect_ibm_db only. No change to the shared base dialect or the ODBC/JDBC implementations, and no SQL rewriting or casts.

This matches the DBAPI's behavior rather than emulating decimal support: ibm_db 3.2.3's Decimal binding path converts the Decimal to its exact string representation and binds it as SQL_C_CHAR; ibm_db_dbi's result conversion produces Python Decimal results. The new raw-DBAPI live test passes before and after the dialect change, independently verifying this behavior for execute and executemany.

Existing Numeric result conversion, including asdecimal=False, is unchanged.

Regression tests

test/test_numeric.py follows the repository's SQLAlchemy testing fixtures/assertions style. It covers DECIMAL and Numeric binding, wide positive/negative values, exponent notation, zero, NULL, ordinary integer/float inputs, result conversion, typed execute/executemany, exact Decimal-valued predicates, and direct DBAPI binding.

Same test file, same Python/SQLAlchemy/DBAPI/server; changing only ibm_db_sa/ibm_db.py between upstream master 7f3866e3a827924055323fd5f10ac40b7a9fb2a7 and this patch:

Tests Before After
Connectionless tests 3 FAIL / 2 PASS 5 PASS
Local DB2 tests 2 FAIL / 1 PASS 3 PASS
Combined 5 FAIL / 3 PASS 8 PASS / 0 FAIL / 0 SKIP

The live failures are exact-comparison failures, not connection/setup failures:

Input Before After
123456789012345.67 123456789012345.7000000000 123456789012345.6700000000
123456789012345678901.1234567890 123456789012346000000.0000000000 123456789012345678901.1234567890
-123456789012345678901.1234567890 -123456789012346000000.0000000000 -123456789012345678901.1234567890

No precision tolerance, rounding, decimal-context adjustment, or cast is used.

Reproduce connectionless tests:

python -m pytest -p sqlalchemy.testing.plugin.pytestplugin --db sqlite -q test/test_numeric.py::TestNumericBind

For the live tests, supply a URL for a disposable database using the existing explicit db2+ibm_db entry point (the SQLAlchemy test plugin expands implicit driver names):

python -m pytest -p sqlalchemy.testing.plugin.pytestplugin --dburi "$DB2_TEST_URL" -q test/test_numeric.py

Validation used Python 3.11.2, SQLAlchemy 2.0.52, ibm-db 3.2.3, pytest 9.1.1, and DB2 Community Edition 11.5.9.0. The SQLAlchemy pytest plugin emits unknown-marker warnings; no tests were skipped. Ruff 0.5.0 check/format for the new test file and git diff --check pass. The entire legacy upstream suite was not run.

Additional live checks

Against the same local DB2 LUW 11.5.9.0 server, a separate SQLAlchemy Core check of connectivity, DDL, reflection, Unicode/quoted/NULL/repeated binds, typed round trips, commit/rollback visibility, dispose/reconnect and pool invalidation passed (8/8). Exact DECIMAL(31,10) round trips (123456789012345678901.1234567890 in and out, as Decimal) were confirmed through each existing URL alias: db2, db2+ibm_db and ibm_db_sa. Released 0.4.0 reproduces the decimal corruption with the same check.

Not covered: other DBAPI or SQLAlchemy versions, DB2 editions/platforms other than LUW 11.5.9.0, ODBC/JDBC, TLS/external authentication. The ibm_db-based AS400 subclass inherits this flag but has not been tested against IBM i.

Signed-off-by: Amin Ghadersohi <amin.ghadersohi@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant