Teacher Observation and Reliability: Additional Insights Gathered from Inter-rater Reliability Analyses

Zepeda, Sally J.; Jiménez, Albert M.

doi:10.31045/jes.2.2.2

Cited by 11 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fourteen are men and seven women. We did an evaluation using Gwets AC1 [22]. Gwet's AC1 can show the level of agreement between two experts.…”

Section: Discussionmentioning

confidence: 99%

“…The number of diagrams used after the validity test was twenty-five student answer diagrams with a data reliability value of 0.959. Since this research aimed to establish a standardized assessment based on expert consensus, we also tested inter-rater reliability [22] amongst the experts. The average measure of interclass correlation is 0.929, based on nineteen experts.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A Different Approach on Automated Use Case Diagram Semantic Assessment

Fauzan¹,

Siahaan²,

Rochimah³

et al. 2021

IJIES

View full text Add to dashboard Cite

The use case diagram is one of the diagrams commonly taught in colleges of computer science. Assessment of use case diagrams is often an obstacle for a teacher in the learning process. It is due to the interpersonal and intrapersonal problems of the teacher in assessing. Interpersonal problems are caused by the absence of an assessment standard among teachers. Intrapersonal problems are caused by the inconsistency of a teacher in assessing many diagrams of student answers. This research aims to create a semantic use case diagram automatic assessment method. Semantic assessment is divided into two kinds, namely property and relationship. All information used is a label translated from the XMI document. Similarity assessment between labels used cosine similarity, employing WuPalmer to perform WordNet searches. The results showed that the proposed method had a substantial agreement with the teacher as an expert; however, a teacher tends to look at property information rather than relationship information to assess use case diagrams.

show abstract

“…Fourteen are men and seven women. We did an evaluation using Gwets AC1 [22]. Gwet's AC1 can show the level of agreement between two experts.…”

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

A Different Approach on Automated Use Case Diagram Semantic Assessment

Fauzan¹,

Siahaan²,

Rochimah³

et al. 2021

IJIES

View full text Add to dashboard Cite

show abstract

“…This can involve using the kappa statistic [37,38,39] or Gwet's AC1 [40,41,42] to measure a method's results and the Alpha Cronbach [43,44] to ensure the data's reliability. But, Gwet's AC1 could be better than the kappa statistic for assessment case [45,46,47,48]. Software reuse papers tested their research using precision and recall, with none utilizing similarity measurements to test their research.…”

Section: What Are the Parameters (Measuring Instruments) Used To Measure The Similarity Between Two Software Products?mentioning

confidence: 99%

Software similarity measurements using UML diagrams: A systematic literature review

Triandini

Fauzan

Siahaan

et al. 2021

regist. j. ilm. teknol. sist. inf.

View full text Add to dashboard Cite

Every piece of software uses a model to derive its operational, auxiliary, and functional procedures. Unified Modeling Language (UML) is a standard displaying language for determining, recording, and building a software product. Several algorithms have been used by researchers to measure similarities between UML artifacts. However, there no literature studies have considered measurements of UML diagram similarities. This paper presents the results of a systematic literature review concerning similarity measurements between the UML diagrams of different software products. The study reviews and identifies similarity measurements of UML artifacts, with class diagram, sequence diagram, statechart diagram, and use case diagram being UML diagrams that are widely used as research objects for measuring similarity. Measuring similarity enables resolution of the problem domains of software reuse, similarity measurement, and clone detection. The instruments used to measure similarity are semantic and structural similarity. The findings indicate opportunities for future research regarding calculating other UML diagrams, compiling calculation information for each diagram, adapting semantic and structural similarity calculation methods, determining the best weight for each item in the diagram, testing novel proposed methods, and building or finding good datasets for use as testing material.

show abstract

“…The basis for administrators' concerns are founded in the respective, but often conflated and inconsistent, purposes and methods of teacher supervision and evaluation (Zepeda & Jimenez, 2019). For example, teacher evaluation can be useful for removing underperforming teachers (Grissom & Bartanen, 2018), however a much larger majority of teachers need a system that provides formative feedback which can be used to improve instructional practices (Mette et al, 2015;Stark et al, 2017).…”

Section: Background and Conceptual Frameworkmentioning

confidence: 99%

A Thirty State Analysis of Teacher Supervision and Evaluation Systems in the ESSA Era

Mette¹,

Aguilar²,

Wieczorek³

2020

ubsw-j-jes.ubiquityjournal.website

View full text Add to dashboard Cite

We analyzed teacher supervision and evaluation policy systems in 30 states since the passage of the Every Student Succeeds Act (ESSA) of 2015 in the United States (US). This qualitative study of state ESSA policy documents and legislation examined how teacher supervision and evaluation systems (TSES) models have been developed under ESSA, specifically regarding how the construction of TSES models conflated formative feedback with summative evaluation. Despite evolving federal-level and state-level education accountability policies spurred by No Child Left Behind (NCLB) in 2001, we argue that TSES systems are influenced by state-level historical political culture (

show abstract

Teacher Observation and Reliability: Additional Insights Gathered from Inter-rater Reliability Analyses

Cited by 11 publications

References 21 publications

A Different Approach on Automated Use Case Diagram Semantic Assessment

A Different Approach on Automated Use Case Diagram Semantic Assessment

Software similarity measurements using UML diagrams: A systematic literature review

A Thirty State Analysis of Teacher Supervision and Evaluation Systems in the ESSA Era

Contact Info

Product

Resources

About