PyTorch hub cache race fix draws review pushback

PyTorch is working on a fix for a long-standing race when several processes hit torch.hub with an empty local cache at once. Concurrent callers of the URL state-dict loader and the hub reload path could each start the same download or extract, which is painful for multi-rank training jobs that share a home or checkpoint directory.

The proposed change takes a file lock around the cold-cache path, extracts archives into a private work directory, then publishes the result with an atomic rename. If a lock cannot be acquired, the code falls back to an unlocked download and warns.

Automated review has blocked merge for now. Reviewers want a declared lower bound on the filelock dependency and packaging sign-off, updated hub docs that mention waiter behavior and new lock files in the cache, and a configurable lock timeout. A fixed one-hour wait is a poor fit for large checkpoints, and default 50 ms lock polling can hammer NFS when many distributed ranks share TORCH_HOME. Lock-timeout tests were also reported to hang instead of failing cleanly.

Until those issues land, multi-process hub downloads remain a best-effort path rather than a guaranteed single-downloader design.