What do allocated and reserved memory measure?
PyTorch reports memory occupied by tensors separately from memory held by its caching allocator. Reserved memory can include reusable space, so a gap between the two is not, by itself, evidence of a leak. See the CUDA memory-management notes.
Why can device usage be higher?
A device-level measurement can include allocations that PyTorch does not manage. A PyTorch allocator snapshot cannot explain every byte on the GPU; communication libraries such as NCCL may allocate memory outside that allocator. Check the snapshot coverage documentation.
How should you compare readings?
- Record the device, rank, step, input shape, and measurement time.
- Compare the same workload phase across repeated steps.
- Use allocation history to investigate persistent growth; do not infer a leak from a single peak.
- Record both memory behavior and throughput after a change.
Clearing unused allocator cache does not release live tensors. Investigate what remains referenced before treating cache clearing as a solution.
What should you investigate next?
Use the OOM investigation guide to collect a reproducible failure. The memory glossary explains the terms used in the measurements.