In machine learning, we traditionally evaluate the performance of a single model, averaged over a collection of test inputs. In this work, we propose a new approach: we measure the performance of a collection of models when evaluated on a single input point. Speci cally, we study a point's pro le: the relationship between models' average performance on the test distribution and their pointwise performance on this individual point. We nd that pro les can yield new insights into the structure of both models and data-in and out-of-distribution. For example, we empirically show that real data distributions consist of points with qualitatively di erent pro les. On one hand, there are "compatible" points with strong correlation between the pointwise and average performance. On the other hand, there are points with weak and even negative correlation: cases where improving overall model accuracy actually hurts performance on these inputs. We prove that these experimental observations are inconsistent with the predictions of several simpli ed models of learning proposed in prior work. As an application, we use pro les to construct a dataset we call CIFAR-10-N : a subset of CINIC-10 such that for standard models, accuracy on CIFAR-10-N is negatively correlated with accuracy on CIFAR-10 test. is illustrates, for the rst time, an OOD dataset that completely inverts "accuracy-on-the-line" (Miller,