On the Reasons for Students' Choices in Rating their Courses: A Preliminary Report

type:学術論文

type:Original Article

As Japanese university students complete course evaluation forms, what are they thinking? Why do students pick a 2, a 3, or a 5? What can we understand about their reasoning? In this study, students from two classes (2010 and 2012) completed two evaluation forms, an official university form, and an instructor-designed form. Each evaluation item had space for students to write their reasons for choosing a rating. Reasons were then be grouped into thematic clusters by common phrases and referents. Results show that evaluation items yielded 6-15 themes in student responses, with mean themes per item ranging from 8.5 to 11.8, depending on instruments and class years. Estimated ratings (or rating ranges) were given to these themes and used to predict actual ratings of the students' responses with said themes. Overall, about 72% of student responses had themes whose Estimated ratings correctly predicted Actual response ratings. At the same time, individual items ranged widely on the proportion of student responses whose ratings could be correctly predicted. Additionally, University and Instructor evaluations differed notably in their patterns of items with high and low prediction rates. Finally, the great majority of student responses - just over 80% - refer to a theme mentioned by only 1-3 others. The findings show an unanticipated variety of themes per item, a predominant tendency toward clarity and reasonability in student responses, and a minority of cases where more than a few students referred to the same theme. These novel and intriguing findings invite a consideration of validity that moves away from statistical associations and toward situated meaning in the classroom context and students'responses toward the events of that context.

identifier:10

identifier:KJ00009866979