sha256 does not convert strings to bytes properly

[cjdelisle]

This is a problem for ever upgrading the sha256 implementation used. As far as I know the actual algorithm is fine but the conversion of strings to bytes silently drops upper codepage characters, doing effectively a string.charCodeAt(i) & 0xff. This was not obvious at first because the sha256 implementation uses an array of integers rather than bytes in order to be faster so the conversion involves some bit-shifting and is not obvious in it's intent.

An actual result of this is there is a possibility for a patch containing UTF to be disguised as one which doesn't, but this is unlikely to happen accidentally and there is no security implication as this sha256 is for integrity only.

[mudassir-jmi]

This is a problem for ever upgrading the sha256 implementation used. As far as I know the actual algorithm is fine but the conversion of strings to bytes silently drops upper codepage characters, doing effectively a string.charCodeAt(i) & 0xff. This was not obvious at first because the sha256 implementation uses an array of integers rather than bytes in order to be faster so the conversion involves some bit-shifting and is not obvious in it's intent.

An actual result of this is there is a possibility for a patch containing UTF to be disguised as one which doesn't, but this is unlikely to happen accidentally and there is no security implication as this sha256 is for integrity only.

Modify the conversion logic to properly handle upper codepage characters and preserve their values during the conversion. You can use language-specific functions or libraries that handle character encoding and ensure accurate byte representation. Avoid operations that mask or truncate higher bits that may contain valuable information.