From Independent Ratings to Communal Ratings

OPEN ARCHIVE

Union Jack
Dannebrog

From Independent Ratings to Communal Ratings

Vis flere oplysninger

Titel: From Independent Ratings to Communal Ratings
A Study of CWA Raters' Decision-Making Behaviours
Forfatter: Lindhardsen, Vivian
Resume: The present maps study maps the decision-making behaviors of experienced raters in a well-established Communal Writing Assessment (CWA) context, tracing their behaviors all the way from the independent rating sessions, where the initial images and judgments are formed, to the communal rating sessions, where the final scores are assigned on the basis of collaboration between two rates. Results from think-aloud protocols, recorded discussions, retrespective reports and reported scores from 20 raters rating 15 ESL essays show that when moving from the independent ratings to the communal ratings, there is little, if any, increase in rater agreement levels and the raters' attention to the textual features corresponding to the official criteria become more evenly distributed. However, rather than consulting the scale descriptors directly in resolving insecurities about score assignment, the raters seemed to rely heavily on each others' expertise, thereby reducing the importance of the scale and emphasizing the value of the community of raters. In validating their scores in the communal rating discussions the raters appeared to be critically and equally engaged in the discussions, and through deliberating and refining their assessments the raters believed that CWA practices produce more accurate scores than in independent ratings and lead to professional development. These interpretations support a hermeneutic rather than a psychometric approach to establishing the validity of the present CWA practices.
URI: http://hdl.handle.net/10398/7743
Dato: 2009-02-23

Creative Commons License This work is licensed under a Creative Commons License.

Filer Størrelse Format Vis
Vivian_Lindhardsen.pdf 8.128Mb PDF Vis/Åbn Vivian_Lindhardsen_phd

Dette dokument findes i følgende samling(er)

Vis flere oplysninger