Evidence at a glance
Turning “It Might Run” into a Searchable Record
Hugging Face’s blog introduces a torch-neuronx model support Space for AWS Trainium. It lets users check whether Hub models run on Trainium without modification and provides the corresponding run script. The tool builds on TorchNeuron, a PyTorch backend that lets Transformers use its own model implementations on Trainium rather than requiring a separate, hardware-specific rewrite for every model.
The Space addresses an engineering question that a general claim of PyTorch support can obscure: having a backend does not mean every combination of model, compiler, and dependencies has been verified. It ties the check to particular models and versions, showing the result, the number of Trainium cores used for sharding, wall time including compilation, and peak device memory. For teams choosing models, that is more useful than a broad statement of hardware compatibility.
Read the Result Alongside Its Test Boundaries
The Space opens on a curated set drawn from models with high Hub download counts or trending scores. Users can switch to Trending or Downloads views, browse Hub rankings by task, and search for a model. Those views include ranked models even when they have not been tested. When a model is deliberately left out, the Space gives a reason: usually that it was not prioritized, though in some cases it is genuinely incompatible. Users can also select the Neuron compiler version, neuronx-cc, and the Transformers version, so “supported” is not a label independent of the software stack.
Each test runs on a Trainium2 instance and shards the model across the smallest number of cores that can hold its weights. Most models use tensor parallelism; mixture-of-experts models without a tensor-parallel plan use expert parallelism only. The current scope is limited to models that fit on at most four Trainium2 devices, or 16 cores. Tests generate 32 tokens: single-core output is compared with a CPU run of the same checkpoint, while a small judge model checks whether sharded output is fluent English. This establishes that a model can run under limited conditions. It does not establish that longer generations will be correct or that throughput will meet a production target.
The Script Connects Compatibility to a Deployment Path
Once a model is selected, the page provides the Neuron stack used for the test, the additional packages to install, and the actual run.py script. The script follows the TorchNeuron recipe: it loads the model, shards it when needed, and generates with a static cache and `torch.compile(backend="neuron")`. A deployment team can run it on a trn2 instance with the same stack, for example using the AWS Neuron Deep Learning Container. When the model is sharded, the script starts its own torch.distributed ranks, so there is no need to launch it separately with torchrun.
The first run compiles the model’s graphs, which is why compilation is included in the reported wall time. The Space also gives a command for fetching graph files built with the matching compiler version from a public NEFF cache bucket: `hf buckets sync hf://buckets/aws-neuron/torch-neuronx-neff-cache/neuronxcc-<version> /tmp/neff_cache`. The result is more than a pass label: it includes versions and an execution path that can help reproduce the test. Reproduction still depends on using a matching compiler and software stack.
Useful for Compatibility Screening, Not Performance Decisions
Support results can also be inspected in the public aws-neuron/torch-neuronx-model-support dataset. It stores Parquet files for combinations of neuronx-cc and library versions, with each row recording the model result, any failure error, the full stack versions, and the script that ran. A team can search through the Space or load the dataset with pandas and filter by model. That makes the evidence from a run available for programmatic review, not just as a status displayed on a webpage.
For now, testing covers only Transformers models for text generation and image-text-to-text tasks. Diffusers, Sentence Transformers, and other Transformers tasks are planned expansions. More importantly, PASS does not indicate acceptable throughput and does not cover longer generations. The practical approach is to use the Space to narrow the candidate list, then measure performance and stability with the team’s own instance, software versions, and workload. If a use case exceeds the tested models, device scale, or generation length, treat the result as a compatibility clue, not a production guarantee.