As the fall semester of the 2013-2014 school year comes to a close, I can now compare my most recent student cohort performance with earlier cohorts. While I am neither beholden to data, nor easily fooled by its misuse, data offers tremendous utility when used properly.
Not all data are the same either. We are all familiar with the phrase “apples versus oranges” when comparing dissimilar items. In my comparisons below, keeping with the fruit analogy, I compare apples with apples. However, one comparison is between a Gala apple and a Gala apple while the others are more like comparing Fuji to Red Delicious apples.
The primary reason for the difference in varietals arises from evolving aspects of my teaching to include specific assessments, grade category weights, assessment retake rules, and etcetera over the past three years. As I mature as a teacher, I tweak various course elements in search of the most effective system to yield maximal student learning outcomes; of course, these differ based on the level of the course, such as algebra 1 versus AP Calculus.
This past year, my tweaks in AP Calculus AB consisted of adopting a no retake policy and changing the weights of assessment and assignment categories, which all honors and AP courses now follow in our department. Additionally, I switched to a “flipped classroom” pedagogical model, requiring students to watch topic-specific videos, to read sections of the textbook, to submit their homework assignment, and to complete a series of online exercises related to the topic, all before providing any in-class direct instruction. The flipped classroom approach is reminiscent of how I learned calculus at West Point via the Thayer Method. However, there were no videos or other online resources available at the time, just me, my Anton Calculus text, and a TI-57 calculator. In keeping with the times, though, at least one instructor at the Academy recently used Khan Academy videos to supplement student learning in his calculus course. All I need to say for members of the Long Grey Line is “The Corps Has…” Much as the grads who learned that we did not need to master the slide rule likely said.
Notwithstanding that last bit of trivia, I believe it is critical to student success, especially in upper level mathematics, that students engage with new material building upon their current schemas before entering a classroom. In this fashion, classroom learning proceeds in a richer, more effective context as students have a framework of understanding which can be further strengthened through group discussions, whole class discussions, or supplemental instruction.
This year’s cohort of students took the same seventy-question, multiple-choice final exam covering the College Board defined AP Calculus AB curriculum for differential calculus as did the earlier two cohorts. Similar to both years prior, students needed every minute to complete the test. Results for the 2013-2014 cohort are encouraging as shown below.
Every statistic for the final exam improved for the 2013-2014 cohort. I am very proud of these students. While it is difficult, if not impossible, to determine causal factors for the increase in performance on the final, I firmly believe that the pedagogical switch served as a strong contributing factor. I also believe most students rose to the challenge seeking supplemental support from many places to include my online resources, tutors, and AP study guides. Also, I worked hard the first two weeks of the semester to dissuade students from taking the course if they were not willing to put in the requisite level of effort. While it may sound harsh, doing so helped students better match themselves with a choice that best suited their interests and abilities. As I learned the past two years, if a student was unwilling to invest effort in the course, there was little I could do to help them understand the material sufficiently enough to pass.
In a perfect world, as some may think, none of these external supports would be necessary. However, those thinkers may not completely understand the complexity involved in teaching mathematics where success on the topics at hand depends significantly upon student mastery and understanding of prerequisite material; I suspect this is true for many other subjects as well. For some reason, most in our education system overlook this essential truth expecting that any and all should attain mastery of the most complex material offered in a public school system, and if they do not, then the system, more often than not spelled T-E-A-C-H-E-R, is to blame. I do not subscribe to this belief. I do believe that any and all should have the opportunity to rise to their peak level in whatever areas they pursue, and I, as a teacher, will show you the most effective paths to reach those heights. However, as we are all unique in our own ways, some students will do exceedingly well in most areas while others will do so only in a few. Such is the diversity of nature and life. More on this to follow in later posts.
As noted last year, when examining various methods of defining performance for my students, this year’s cohort continues to perform on par with last years’, if not better on some metrics. Each of the following figures takes into account student performance for the entire semester, which, for the 2011-2012 (“2011”) cohort and the 2012-2013 (“2012”) cohort, is computed as a weighted average where assessments comprise 60% and assignments 40%; for the 2013-2014 (“2013”) cohort, weights for assessments comprise 80% and assignments 20%. The department switched to this weighting this year for all honors and AP courses.
The first figure below depicts scores in 10% bins, akin to a traditional grade scale. The second uses my rating scale where each bin consists of a 15% span (e.g. Advanced consists of scores from 85-100%). The third figure ascribes letter grades to the ratings following my grading scale where the primary change is the name assigned to the bin (e.g. Advanced becomes an A) coupled with upward adjustments made for students who are within striking range of the next grade level. Given the uncertainty in assessments, I believe it is only fair to adjust upwards when a student is within two- to three-percentage points of a cut score.
For this cohort, I show scores below 60% in 10% bins, rather than aggregating them into one bin for all scores less than 60%. Doing so shows that this year’s cohort includes a set of students with scores in the 40-49% range. Also, there are no students whose total score places them in the 90-100% range. Each of these differences likely stems from: 1) the fact that retakes were not offered this year given our new departmental policy as well as 2) a shift in assignment frequency and scoring, where this year every homework assignment counted, as opposed to prior years where they either did not count at all (2011-2012) or were announced beforehand (2012-2013) and assignment weightings this year were reduced from 40% to 20% of the semester grade. Also, for whatever reason, a handful of students this year submitted no homework or very few, even though I accepted them for up to 75% partial credit up to the last day of the course. As the following images show, these changes did not necessarily impact students’ ability to receive an A in the course.
The distribution of student understanding in differential calculus is captured in the image above. Similar to the semester scores, students in this cohort who rated below basic would likely have ended up scoring in the basic range if retakes were permitted. However, the department elected not to allow them starting this year.
Lastly, student grades this year included a small set of Ds. These students will have the opportunity to raise their grade in the second semester as well as after taking the AP Exam if their score is a 3 or higher. All students will have the opportunity for their course grade to be revised upward if their AP Exam score warrants. Specifically, an AP Exam score of 3 corresponds to a C in the course, a 4 to a B, and a 5 to an A. No students last year ended up with a higher grade as a result of their AP Exam score, however.