Judicial Performance Evaluation (JPE) is generally seen as an important part of the merit system, which often suffers from a lack of relevant voter information. Utah's JPE system has undergone significant change in recent years. Using data from the two most recent JPE surveys, we provide a preliminary look at the operation of this new system. Our results suggest that the survey component has difficulty distinguishing among the judges on the basis of relevant criteria. The question prompts intended to measure performance on different ABA categories are also indistinguishable. We find evidence that, on some measures, female judges do disproportionately worse than male judges. We suggest that the free response comments and the new Court Observation Program results may improve the ability of the commission to make meaningful distinctions among the judges on the basis of appropriate criteria.