Research shows worth of collaborative student assessment
By Trevor Pritchard
Ever feel that you knew more about a subject than your grade reflected? That you couldn't possibly have known about all of Shakespeare's plays when you were only asked about Othello?
Or, on the other hand, have you ever felt that your students should know more about cellular respiration than their exam marks suggested?
It's conflicts like these that Computer Science PhD student Diego Zapata-Rivera is attempting to remedy.
Zapata-Rivera's thesis, Learning Environments Based on Inspectable Student Models, attempts to bridge the communication gulf between students and teachers by using computer-based student models to introduce collaborative methods of student assessment into the classroom.
"Imagine students trying to learn [independently]," he says. "They are reading the material, but not interacting with it. There are no questions, no guidance or support for reflection."
What Zapata-Rivera hopes the future will bring are e-learning environments which support student reflection, critical thinking, and more accurate methods of student evaluation, and which would be accompanied by a variety of tools designed to help students and teachers interact with these environments.
Called inspectable student modeling tools (ISMTs), these systems allow the students to express their opinions about the teacher's grading methods. Students can interact with ISMTs and provide feedback about whether the methods adequately reflect and assess their own levels of knowledge.
Some of these ISMTs take the form of artificial animated characters. "One of them is called Doņa Rita," remarks Zapata-Rivera. "She's an animated schoolteacher who helps students interact with these models. If Doņa Rita finds out that [there is a] difference between the perceived knowledge of the students and the teachers, she can ask the student to elaborate, to explain why he or she thinks they know more or less [than the tests show]. It encourages reflection."
Other ISMTs used in Zapata-Rivera's thesis include VisMod, ConceptLab, and SMV, each of which aid in the assessment process.
"In the traditional student environment," Zapata-Rivera continues, "students cannot change or update the results of quizzes and tests. [They] are not even asked to give their opinions on their own assessment. Using learning environments based on inspectable student models, the teacher is not the only one responsible for student assessment."
In November 2002, Zapata-Rivera created The Learning Game, a Java-based program designed both to measure the degree of student reflection and to examine the accuracy of the inspectable student models used in this particular learning environment. Students from Computer Science 111 were asked to play the game, in which they were asked to interact with the ISMTs and update their level of knowledge about Java programming. They would then offer feedback designed to make the student model more accurate.
The results showed that students and teachers each have information 'gaps' which can hinder the evaluation process. "The teacher doesn't have all the information he or she needs, and the students don't [always] judge their knowledge in an accurate way," says Zapata-Rivera, "and each [viewpoint] can have problems if they are considered separately." However, he clarifies, when the student model integrates evidence from different sources, from both the student and the educator, it becomes more accurate.
Two years ago, Zapata-Rivera carried out an exploratory study using ISMTs in his hometown of Caldas, Colombia, at the same school he attended as a child.
"I went home to Colombia two years ago to test this program at the school in which I studied. I was able to talk to my former teachers and see how the program was working."
The outcomes from Zapata-Rivera's studies have been published at several international conferences, an accomplishment regarded highly by his supervisor, Dr. Jim Greer.
"Diego has been the ideal graduate student," says Greer. "One paper was voted the best paper at a major international conference in 2001 and another was a finalist for best paper at the same conference in 2003. His work [with] intelligent learning environments is becoming influential worldwide."
Despite the academic optimism surrounding his project, Zapata-Rivera admits that there are still obstacles to implementing this collaborative assessment approach. For instance, sometimes students do not feel confident about their own assessment and may exaggerate their own abilities to appear more knowledgeable. However, the different sources of information used in The Learning Game successfully addresses this issue, Zapata-Rivera says.
As well, sometimes students assume that teachers, because they are in a position of authority and are usually older and more experienced, are infallible. Because of this mindset, students can feel unqualified to offer their personal assessment of their levels of knowledge.
Not true, states Zapata-Rivera: "Some students think teachers are the ones who know about everything. They're used to thinking that some negative action will be taken if they answer 'I don't know,' or 'I don't remember'."
"But not remembering shouldn't be seen as a negative answer in these systems. Once you realize you don't remember, it's a really good opportunity to begin learning about it!"