This is the report of the third phase of a multi‐phase project designed to develop an inventory of documented accomplishments for graduate student selection. The material developed in earlier stages was pilot‐tested in Phase Three in 26 departments that represented the fields of English, biology, and psychology. The major purpose of this study was to investigate the instrument developed in Stage Two in terms of: 1) technical soundness, and 2) the feasibility of its use in the admissions process. This study represents the transition of the use of documented accomplishments to predict graduate school achievement from a research phase to a more operational phase during which the instrument would be tested in actual admissions situations. The departments were asked to administer the instrument to their newly enrolled graduate students. Straightforward descriptive and correlational analyses of the responses to the inventory were conducted to: (1) describe characteristics of the students and the instruments; and (2) identify the most reliable clusters of items and indices of accomplishment. Students were followed up at the end of their first year to assess their graduate school accomplishments and their relationship to previous attainments. This information provided data for analyses of the short‐term correlates of the measures. In addition, faculty were interviewed to determine the plausibility of the content of the instrument and the most accurate and operationally simple instructions and reporting formats. Students' reactions to the inventory were obtained by interview and by additional questions at the end of the inventory.

Educational Testing Service is currently developing a new generation of teacher assessments‐The Praxis Series: Professional Assessments for Beginning Teachers™. The assessment series consist of three separate, but related, components. Praxis I: Academic Skills Assessments will assess the candidate's command of basic academic or enabling skills in reading, writing, and mathematics. Praxis II: Subject Assessments will test the candidate's grasp of subject matter and his or her knowledge of the teaching and learning process. Praxis III: Classroom Performance Assessments will assess the candidate's application of this knowledge in an actual classroom setting. This document describes a series of formative studies that were conducted in support of the development of Praxis III. The research efforts were targeted in three broad areas: (a) field‐testing of the various data‐collection instruments; (b) examination of the processes and strategies involved in retrieving, coding, and evaluating teacher performance data; and (c) analysis of how the performance assessment addresses issues of diversity in teaching and learning. The overarching goal of these studies was to identify strengths of the performance assessment system as well as aspects that needed further refinement. The studies were conducted in Minneapolis, Minnesota and Dover, Delaware during November and December 1991. Trained assessors working in pairs carried out an assessment cycle in which they observed a candidate teaching a lesson, interviewed the candidate before and after the observation, and reviewed several documents the candidate had completed. A total of 18 candidates were evaluated. The assessors took notes during the interviews and observations, and then coded them. From their coded notes, assessors selected pieces of evidence to include on a Record‐of‐Evidence form, a document that summarizes the evidence the assessor obtained for 21 criteria of good teaching and provides a rationale for each rating. Assessors weighed the evidence they obtained for each criterion and used a scoring rule to assign a rating on the criterion. When the assessors had finished rating candidates, they met as a group to evaluate the assessment system. The assessors completed questionnaires and work sheets and engaged in small‐ and large‐group discussions to share their reactions to using the assessment system. These activities and the records of evidence provided the data for the formative studies. This overview document highlights the major findings from each of the formative studies and discusses the implications of those findings for the Praxis III assessment system. The last section of the paper describes how the developers used the results of the formative evaluation to guide them in making a number of informed changes in Praxis III, that is, in revising the domain descriptions, criterion descriptions, and accompanying scoring rules. Changes were made in the data‐collection instruments and in the assessor training program, and new procedural guidelines for carr...

No abstract

No abstract