Skip to content

Commit a764742

Browse files
cary-ilmmusicinmybrain
authored andcommitted
Fix signed integer overflow in LossyDctDecoder_execute() pointer arithmetic (#2329)
`numBlocksX` and `numBlocksY` are declared as `int`. Two pointer-offset expressions in `LossyDctDecoder_execute()` multiplied them as signed 32-bit integers before using the result as a pointer offset: rowBlock[comp] = rowBlock[comp - 1] + numBlocksX * 64; currDcComp[comp] = currDcComp[comp - 1] + numBlocksX * numBlocksY; `dataWindow.max.x` is a signed 32-bit value in the EXR file format, so `numBlocksX` can reach `(INT32_MAX + 7) / 8 = 268,435,456`. At that point `numBlocksX * 64 = 17,179,869,184` overflows `int32`, and `numBlocksX * numBlocksY` overflows even sooner. The wraparound produces a small or negative pointer offset, causing `rowBlock[comp]` and `currDcComp[comp]` to point into already-used or pre-buffer memory rather than the intended component stride. Fix: cast `numBlocksX` to `size_t` before multiplying so the arithmetic is performed in pointer-sized unsigned arithmetic: rowBlock[comp] = rowBlock[comp - 1] + (size_t) numBlocksX * 64; currDcComp[comp] = currDcComp[comp - 1] + (size_t) numBlocksX * numBlocksY; This is consistent with the allocation on the line above, which already uses `(size_t) numComp * (size_t) numBlocksX * 64 * sizeof(uint16_t)`, and with the packed-DC count check, which uses explicit `uint64_t` casts. Made-with: Cursor Signed-off-by: Cary Phillips <cary@ilm.com>
1 parent d578381 commit a764742

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

pxr/imaging/plugin/hioOpenEXR/OpenEXR/OpenEXRCore/internal_dwa_decoder.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,7 @@ LossyDctDecoder_execute (
265265
}
266266

267267
for (int comp = 1; comp < numComp; ++comp)
268-
rowBlock[comp] = rowBlock[comp - 1] + numBlocksX * 64;
268+
rowBlock[comp] = rowBlock[comp - 1] + (size_t) numBlocksX * 64;
269269

270270
//
271271
// Pack DC components together by common plane, so we can get
@@ -275,7 +275,7 @@ LossyDctDecoder_execute (
275275

276276
currDcComp[0] = (uint16_t*) d->_packedDc;
277277
for (int comp = 1; comp < numComp; ++comp)
278-
currDcComp[comp] = currDcComp[comp - 1] + numBlocksX * numBlocksY;
278+
currDcComp[comp] = currDcComp[comp - 1] + (size_t) numBlocksX * numBlocksY;
279279

280280
for (int blocky = 0; blocky < numBlocksY; ++blocky)
281281
{

0 commit comments

Comments
 (0)