freenode
AI & ML

PyTorch CUDA adaptive max-pool backward can corrupt device memory

Unchecked out-of-range indices in the backward kernel become atomic-write destinations, unlike the CPU path which raises an error.

A PyTorch bug report says the CUDA implementation of adaptive max-pool 2D backward will write past its output buffer when given out-of-range index values, silently corrupting GPU memory instead of raising an error.

The backward operator is supposed to scatter gradients using indices produced by the matching forward call. The forward documentation states that contract, but nothing at the backward entry checks that the values actually fall inside the destination buffer. On CUDA, each index is used directly as the address of an atomic add. Fill the index tensor with values such as 100 or the top of the signed 32-bit range and those atomics land far outside the allocation.

NVIDIA's compute-sanitizer reports an invalid 4-byte global atomic and names the CUDA adaptive max-pool backward path. A bare run with launch blocking enabled ends in a CUDA error at synchronization. The CPU backend, given the same out-of-range indices, raises a clean error.

The report was filed against PyTorch 2.13.0 built with CUDA 12.6, on Ubuntu with an RTX 3090. The expected fix is to validate index values, or at least bounds-check inside the kernel, so the CUDA path fails the same way the CPU path already does.