As I explained towards the end of my latest post, Measures of Math Lab Learning Outcomes, students in my math lab course were provided the answers to their spring final exam the day before their final. There were several reasons for my doing so, which I detail in the post.
I also speculated that a student’s score on the final exam did not measure their learning as much as it measured the contribution of their short-term memory, at least in certain cases.
The data series titled “Spring Final Exam” is shown with blue circles in the following figure. This data series reflects student scores on the final exam on a correct versus incorrect basis; that is, whether a student chose the correct multiple choice response irrespective of whether they knew it was correct or not.
The adjusted final exam scores, titled “Adjusted Spring FE,” is in maroon, using squares for each data point. This data series reflects whether a student understood how to arrive at the correct answer in some fashion. The adjusted data were determined by evaluating student written work on the final exam. If their work on a specific question demonstrated understanding, their correct result counted. On the other hand, if their work was grossly incorrect, their result on that question was no longer counted as correct. If they did not show work for any question, their correct result counted if their answer on the same assessment from one month earlier, titled “Post CST Assessment” in the figure, was correct; they did not have to show their work on the earlier assessment.
As can be seen in the figure, the adjusted spring final exam scores (maroon data series) mostly differ from the initial spring final exam scores (blue data series) by ten percentage points or so; the average difference between the two data series is twelve percentage points. A ten percentage point difference in scores translates into three questions on the assessment.
For a few students, however, the difference between their “correct / incorrect score” and their “understanding score” exceeds fifteen percentage points and for one student equals forty-six percentage points, as the following figure illustrates.
The Hypothesis (aka Speculation)
I believe that the difference between the two data series is a rough indication of a student’s short-term memory; more precisely, I believe it indicates the contribution to their score as a result of their ability to remember the specific answer to a question when they did not truly understand how to arrive at the answer. Many students showed work for questions for which they remembered the correct answer, however, their work did not make sense mathematically. Whether my hypothesis is true or not is difficult to prove at this point. I’ll ponder my statistics to see if a t-test makes sense here or not. My gut tells me to measure the difference between the average of the fall semester final exam score, the pst CST assessment score, and the adjusted spring final exam score on one hand and the spring final exam score (unadjusted) on the other hand. It seems that difference may be a better indicator of short-term memory versus understanding.
Growth in Understanding
Speaking of understanding, the following figure offers a perspective of each student’s growth in mathematical understanding after taking the math lab course.
It is clear that many students improved their performance on this assessment over the school year with forty percent of students improving by twenty percentage points or more. At the same time, four students barely budged in their understanding. I plan to reflect further on those students to see what might strike me about their minimal progress. Six students who finished the class either did not start the class with me, or missed the diagnostic test. Hence, it is more difficult to figure to what extent they improved, or not. Nonetheless, it is abundantly clear that one of them turned on the gas at the end of the semester, as his / her score improved considerably, and with true understanding; it was not a fluke or an aberration. The five other students’ individual scores clustered together fairly tightly within a limited range; hence, I believe they did not improve in any large measure during their time in the class, which for four of them was a little over six to eight weeks.