Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.

This post walks through system-level profiling of inference on AWS Trainium using TorchNeuron, its open source native PyTorch backend, starting from scratch with no prior profiling experience. It shows how to configure torch.profiler with NeuronConfig and NeuronProfiler to capture a clean trace, read the key_averages() table to find operations consuming Neuron device time, and follow an operation through Neuron Explorer from framework dispatch to hardware execution. Using three RMSNorm implementations, it reports naive ops at 2,355 µs of Neuron time versus 403 µs for torch.rms_norm and 376 µs for naive plus torch.compile.