Memory Analysis Engineer, XScope

About the role

XScope explains memory instead of just measuring it: every byte maps back to the tensor that owns it and the source line that created it, and a few minutes of profiling catches doomed configurations before launch. You will own that attribution engine and the preflight check behind it.

What you will do

• Attribute GPU and CPU allocations to tensors, modules, and source lines across PyTorch versions
• Model peak memory for FSDP, ZeRO stages, tensor and pipeline parallelism from single-node profiles
• Diagnose retention: which object holds a tensor alive across steps, and why
• Turn findings into recommendations: parallelism layout, micro-batch, activation checkpointing
• Ship reports that an ML engineer can act on without reading the code

What we are looking for

• Deep knowledge of PyTorch internals: autograd, the caching allocator, distributed wrappers
• Experience building or extending memory profilers, allocators, or leak detectors
• Strong Python and C++
• Have run large models yourself and hit OOM the hard way

Nice to have

• Published work on memory analysis or allocators (ISMM, OOPSLA, PLDI)
• Experience with Megatron-LM or DeepSpeed internals