We describe a software simulation test bed for evaluating Concepts of Operation (CONOPS) for Unmanned Aircraft Systems (UAS) flying earth science missions. The Mission Operational Concept Evaluation Framework (MOCEF) aids in the rapid evaluation of proposed system automation designs, including intelligent controllers for vehicles, sensor payloads, and decision support systems, on a wide range of missions. Such broad evaluation is prohibitively expensive when limited to physical experiments and real missions. MOCEF allows evaluation of automation concepts in multiple mission scenarios operating in a wide range of environments. It records mission metric parameters such as the quality of sensor data obtained, flight time, stress on the vehicles, and air traffic control rules infringed or invoked. This information can be fed into specific mission metric formulas to rate performance and into the Google Earth tool for visualization.