in “framing the question”
- Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., … Mong, C. (2007). Using Peer Feedback to Enhance the Quality of Student Online Postings: An Exploratory Study. Journal of Computer-Mediated Communication: JCMC, 12(2), 412–433.
“In addition to the benefits of receiving adequate feedback, students may also benefit from giving peer feedback.”
This “exploratory” study sets out to determine if requiring peer feedback in an online graduate course will lead to an improvement in the quality of online discussion. Study participants were required to respond to discussion questions online, and then score each other’s postings using a scale based on Bloom’s Taxonomy. According to the authors of the study, “quality was maintained” in the student discussion postings; however, the hoped-for improvement of the quality of the discussion postings did not occur. The premise for the study was based on “Black (2005), [who stated] most online discussions consist of sharing and comparing information, with little evidence of critical analysis or higher order thinking.” The authors set out to explore whether or not peer feedback could “promote these higher levels of thinking.” The feedback students gave, however, was in the form of grading, and not in any sort of discussion-engendering format. Students “provided feedback to each other, specifically related to the quality of their postings” on a Bloom’s Taxonomy scale. The study had “the expectation that this would enable them to grow and learn from each other, and thus, to co-construct knowledge and understanding (Roehler & Cantlon, 1997).” Although there is little evidence in the study that the students grew and learned from each other and co-constructed knowledge and understanding, and although the discussion postings themselves did not improve after the implementation of peer feedback, the student perceptions of the worth of giving and receiving feedback is worth discussing and gives merit to the overall study.
Although the study design did do what it said it was going to do — require students to post online responses to discussion questions, require students to give feedback to each other’s postings, and evaluate if the quality of their postings improved over time — the focus of the study seemed to be more on the perceived benefits of giving and receiving feedback rather than the quality of the discussion postings. In fact, most of the article discussion focused on student perceptions of giving and receiving feedback. In addition, when students did respond about the quality of their own discussion postings, their responses were focused on “us[ing] words to help them [the student scorer] see” and it was “useful to know what the person would be looking for.” This doesn’t speak to higher quality discussion postings or higher order thinking at all; the students were not discussing their writing through the lens of higher order thinking but rather through the lens of the grade. This speaks to giving the graders what the graders want to see in order to get a good grade. That is not a higher order thinking skill, but rather a knowledge, comprehension, and application skill (worth a 1 on the rubric.)
Unfortunately, the discussion questions themselves weren’t asking for higher order thinking (and the researchers do later acknowledge this), so it seems disingenuous to grade a student — or ask students to grade each other — on a scale that asks them to do something that the questions themselves weren’t asking students to do. Also, the discussion questions did not actually engender discussion, or if they did, this was not discussed in the study. The initial framing of the study was that online discussions are lacking in substantive content…but really, this study wasn’t asking students to discuss anything. They were crafting a detailed answer to an instructor-posted question and framing their answer in a way that a student grader could score it. This is the opposite of “discussion.” The addition of earning a grade for each response on Bloom’s Taxonomy would also hinder open and thoughtful discussion, not promote it. As a student, if I know that any response I give will be scored on a rubric, I am more apt to simply give the required number of responses and then fall silent, rather than risk my grade by posting less grade worthy responses for the sake of discussion. Grading every response turns the focus onto the grade and takes the focus away from the promotion of thoughtful discussion.
The Bloom’s Taxonomy-based rubric, ironically, was asking for the feedback-givers to use higher order thinking skills (evaluation, worth 2 points on the scale) but it would not have engendered higher scoring discussion postings, unless each discussion question specifically asked the students to analyze, synthesize, and evaluate (2-point answers). The majority of the discussion questions were asking for 1-point answers (knowledge, comprehension, and application). Non-substantive comments received zero points on the rubric. But, because there would be no benefit to students to post non-substantive comments, and because the discussion prompts themselves were asking for knowledge, comprehension, and application, there was really no where for the discussion postings to go to score higher; likewise, students would not suddenly start posting non-substantive answers partway through the course, so the scores would not logically drop, either.
I struggle with the premise and perceived outcomes of this study. As a k-12 teacher, I often have students score each other’s writing and give each other substantive feedback related to “bump scores” (how to move up on the rubric) and to recognize what they did well to earn the scores they earned. Not only does this help students see through a reader’s lens, but it helps them in critiquing their own writing. However, student grading can never be a part of the actual grade; it would be unethical for students to actually grade each other. Instead, it’s the act of giving feedback that earns the grade, and not the initial piece of writing. As a means of formative assessment, I am checking for student comprehension of the elements of writing that are effective and substantive. I am grading the feedback, not the essay. And, as a classroom teacher, formative assessment must be timely, and the entire purpose of it is to check for understanding and instruct teaching. In this study, the assessment grades took weeks to get back to the students, and the grades did not influence or instruct the teaching whatsoever. Although the authors discussed the importance of feedback as formative assessment, the design of the study did not allow the feedback to function as formative assessment.
In essence, this study should have been titled, “Will requiring students to give feedback enhance the quality of the feedback they give?” By reframing the question, the authors could have precisely focused in on the benefits of giving and receiving peer feedback instead of getting lost in the periphery discussions of the quality of online postings, formative assessment, and Bloom’s Taxonomy.
sharon murchie 