Optimizing fertilizer use, such that yield is maximized while input and environmental costs are minimized, is a key goal of sustainable agricultural development. In permanent grassland systems, biomass production increases under fertilization on average, but information is lacking about variability in grassland nutrient limitation, posing risks of underyielding in some areas and overfertilization in others. Here, using a globally distributed fertilization experiment, we show that spatial and temporal variability in grassland nutrient limitation is extensive across 66 sites from six continents. While 90% of sites displayed average increases in biomass production with fertilization, 26% of all experimental units (plots*years) were non-responsive to fertilization and only seven sites displayed ubiquitous positive responses. Importantly, this was not driven by the duration of fertilizer application: there was no difference in the probability of non-responsiveness between periods of short- and long-term application. Similarly, this effect was not driven by site- or block-level limitation and ceasing fertilization of sites or blocks that on average were nonresponsive would have only reduced overapplication to 23% or 18% of experimental units, respectively. Instead, temporal variability in aridity and spatial heterogeneity in initial productivity were the main factors explaining variability in nutrient limitation. Our results suggest that using these variables to guide fertilizer decisions could reduce overfertilization. Overall, our work demonstrates substantial spatial and temporal variability in grassland nutrient limitation while clarifying opportunities for the precision management of yield in global rangelands.