Measures of Math Lab Learning Outcomes

This past school year, AY 2012 – 2013, I taught an algebra 1 intervention course to approximately twenty-five freshman. [1]   The course, titled “Math Lab,” served to support students who scored below basic or far below basic on their prior year algebra 1 California Standards Test (CST). [2]

Purpose for Math Lab

Students are placed in a math lab course concurrently with algebra 1 with the belief that the interventions in math lab will enable them to pass algebra 1.  Unfortunately, when a student is significantly behind in their prerequisite skills and knowledge, especially in mathematics, the path to passing a course reliant upon mastery of the prerequisites is not always short, nor successful.  Notwithstanding the cold, hard reality that plays out for many students far behind in their understanding of, and proficiency with, mathematics, it is imperative that they be continuously encouraged in their efforts to learn and master the prerequisites, and ideally, the prescribed standards of the course.  An excerpt from the Mathematics Framework for California Public Schools, Kinder­garten Through Grade Twelve underscores this sentiment.

“… some students who need remediation perceive their low abilities to be unchangeable, expect to fail in the future, and give up readily when confronted with difficult tasks. Their continued failure confirms their low expectations of achievement, a pattern that perpetuates a vicious cycle of additional failure. What are needed are instructional programs that create steady measurable progress for students, showing them that whatever difficulties they might have had in the past, they are learning mathematics now.”  (California State Board of Education, 2005)

For those of you who know me, I passionately work to give students the opportunity to experience growth in their knowledge, skills, and understanding repeatedly throughout a course; I teach, assess, reteach, assess, and re-reteach until I believe a number of students have grasped the requisite concepts and/or procedures before moving on in the curriculum; I am not a fan of pacing calendars, although there comes a time when one must move on or there is insufficient time to cover the most important standards.  [3]  At the same time, the supposition in the excerpt above that an instructional program, per se, creates steady measurable progress for students ignores the student’s contribution, or more precisely whatever continued difficulties they may have with learning the course content.

Teachers demonstrating optimism, speaking positively, and setting high expectations can help students to overcome low self-images of their mathematical abilities. [4]  Yet, the blind application of this approach can lead to missing very real issues within a student that need to be addressed.  Furthermore, it may not be possible to mitigate a student’s concerns in the confines of the limited, one-on-one time available between a student and a general education teacher, or even with whole-class encouragements, as I often give.  Even then, one must consider that a case might arise where a student is so challenged that they need separate, more intensive interventions, which no general ed teacher could realistically offer in the current educational system we call public high school.  Yet, we face situations like this often where we can only do so much given our limited resources.

Real Learning Outcomes Data for a Math Lab

As an example, at the end of the first semester of the 2012-2013 school year I wrote in Black Cloud, Silver Lining that my math lab students, as a whole, improved their performance on a diagnostic test by 15 percentage-points moving from a 40% to a 55% average score by the end of the semester.  While most of them remained at a below basic level of understanding on essentially a sixth-grade level assessment, 80% of students nonetheless improved their scores; two students attained improvements of 40 percentage-points or more!  The small wins are worth celebrating, and the larger wins even more so.  Accordingly, I shared the graph below with students after each pass at the assessment, except after the spring final exam, as the school year was over.

2012-13 ML Growth EOY

Unfortunately, the net improvement from first semester to second semester was a meager two percentage-points using the data for students present both semesters.  Factoring in new students scores, there was no growth in the second semester.  As the data series labeled “Post CST Assessment” reveals, half of the nineteen students who took both “end of semester” assessments scored lower this time around. [5]  My primary hypothesis for the decline in the rate of improvement follows.

Algebra 1 Pace Too Fast for Many

In the second semester, I switched from reteaching the foundations of arithmetic to reteaching the algebra 1 concepts and procedures present in most second semester algebra 1 courses; this is the most typical implementation of an algebra 1 intervention course when run concurrently with an algebra 1 course.  As most know, second semester algebra 1 content is vastly more challenging than first semester as it address non-linear concepts such as distributing and factoring polynomials as well as solving quadratic equations, rational equations, and radical equations.  It became clear quite rapidly that while students benefited somewhat from having an extra period with which to engage new algebra 1 material, the rate at which they needed to work in the math lab class far exceeded their collective ability.  In fact, the rate in the algebra 1 sections exceeded most students’ ability given their understanding and skill levels entering the course, as well as throughout first and second semester; more to be written about that issue in a future post.  Hence, math lab students did not spend much time on arithmetic second semester, and the time they spent on algebra did not increase their prerequisite understanding much, if at all, as we needed to keep pace with the algebra 1 classes.

My secondary hypothesis relates to my first, with the following refinement: as we spent less time on arithmetic skills second semester, and their use in the second semester came secondary to understanding newer, more challenging algebraic concepts, many students actually regressed in their ability to recall and apply the arithmetic skills we learned first semester.  In some ways, this data series could be a proxy for long-term learning retention, or the ability to recall and apply learning from over four months earlier.  I am intrigued by this potential linkage to long-term memory.  At the same time, the same percentage of students either remained at, or improved in, their level of understanding, a few by twenty-percentage points or more.

Measures of Success

The following table, along with the earlier graph, summarizes different measures of math lab success.  Most students passed math lab, albeit some by the skin of their teeth coupled with my extremely liberal grading scale that uses 15% point bins for grades A through D.  You need to have less than a 40% overall in the course to receive an F.  [6]   Forty-percent of students received an A or a B in math lab in each semester; this includes the – and + varieties of letter grades.

2012-2013 Pass Rates

While most students passed math lab, many less passed algebra 1, especially second semester; a large percentage passed first semester, albeit 65% of those who passed did so with a D-, D, or D+ and only 8% achieved higher than a C+.  These are not miracle stories.  However, many students worked diligently to overcome a significant deficit in their understanding of basic mathematics.  I encouraged them along the way and they encouraged me.  We accomplished a lot first semester, which I came to see when reflecting upon student results on the fall final exam.

As I discussed earlier, second semester algebra 1 is much more challenging than first semester.  While the arithmetic skills students improved upon in first semester math lab served them well in first semester algebra 1, those skills were dwarfed by the more challenging non-linear concepts in second semester.  Thus, while math lab provided all of my students the opportunity to experience success, not all achieved success; yet, for most, it was not for the lack of trying.

Is Math Lab Ultimately Helpful?

If a significant percentage of math lab students did not pass their second semester of algebra 1, is the intervention course a waste of time?  I believe that depends upon whether or not the intervention course, to the greatest extent possible, is meant to improve students mathematical skills from the level when they entered the course or to enable them to pass algebra 1 irrespective of where they started the year in their prerequisite understanding.

If it is the latter, which implies success with the former, then I believe math lab is not helpful enough for many students.  I am not sure if it ever will succeed for all given how far behind some students are in their prerequisite skills and understanding.  If it is the former, I believe it is supremely helpful for many, but not all, students.  As I mentioned earlier, some students are so challenged that they need more intensive intervention than a general education course provides, even taking an Individualized Education Plan (IEP) or a 504 plan into account; forty-percent of my students were on one of these plans, seventy-percent of those had an IEP.  Unfortunately, public education today is woefully under-resourced in its ability to offer enough support to enable all students to overcome their learning difficulties, in spite of differentiation or other modifications.  We have a great system by many measures.  At the same time, if all students are to meet mandated levels of proficiency before they graduate high school, then much more investment is required for those yields to be considered remotely possible.

EPILOGUE

As many may wonder, I have yet to speak to the data series labeled “Spring Final Exam.”  I reserved discussion of that data set until the end, as I hypothesize that it does not measure learning as much as it measures short-term memory, at least in certain cases. [7]

Why do I say this you ask?  As we approached the end of the semester, I considered what would best serve as a final exam for my math lab students.  After reflecting on this for a while, I settled upon using the same diagnostic test for a fourth and final time. [8]  This enabled me to continue supporting my math lab students with preparations for their algebra 1 final exam without burdening them, or me, with a separate series of review sessions.

Additionally, I decided to provide my students with the answers to the questions on the diagnostic test, unlike any earlier time.  [9]  The day before their final, I worked each question orally, without showing the work for any solution; afterwards, I noted the answer on the white board next to the question.  I also told students at the outset that:

1) this was going to be their final exam,

2) they could copy anything from, or even take a picture of, the white boards,

3) they should take notes of my oral work,

4) they should study their notes that night,

5) they could overcome a failing grade in the course if they passed the final exam,

6) they needed to show correct work on the final exam to receive credit, and

7) they could not use their notes on the test.

The last two points caused great consternation given the groaning and pleading that ensued.  However, my students knew me all too well, and when I made up my mind, no amount of cajoling on their part could make me change it. [10]

The two photos below show all the questions on the assessment with the correct answers.

2013-06-03 10.16.16

2013-06-03 10.16.43

Not so surprisingly, student scores this go around were significantly higher than earlier times; the average score improved by twelve percentage points to 69%; three students even scored 90%.  One student improved fifty percentage points from their recent “Post CST Assessment” score, all in the matter of six weeks!  Yet, as noted in [7] below, a key point to keep in mind for this data set is the fact that these scores have not been adjusted to account for student understanding, or lack thereof.  At the same time, students who knew they were failing before the final exam knew they could pass the course if they passed the final exam, as I describe further in [8] below; hence, some students were highly motivated to prepare for the final exam.

As you may infer from my “all in the matter of six weeks” phrase above, I am highly suspicious of near astronomical results in such a short period, especially for students of whom I know struggle mightily with many of the most basic concepts and procedures, such as the subtraction of integers.  At the same time, I am pleased that these students invested time in studying for the final exam.

When I post next about this data set, I will address my hypothesis as to student scores on the spring final exam, both in their current form where they are scored simply correct or incorrect as well as when they are adjusted for the correctness of student work.

Until then, if anyone has questions about my commentary, the data, or anything else about this post, please let me know in the comments section below.

**********************************

NOTES:

[1]  Twenty-three students started the course.  Three joined midway, two of those students left before the end of the semester along with one from the initial cohort.  Another two left at the end of the first semester.  Four new students joined throughout the second semester and one student who left first semester returned leaving twenty-five students enrolled in the course at the end of the second semester.

[2] All but one of the math lab students took algebra 1 in the eighth grade.  The average CST score for the section was just above the cut score delineating below basic from far below basic.

[3] A set of pre-defined standards for a course is a blessing and a curse.  It provides the basis for a high-quality course, where specific content knowledge, skills, and understanding are taught.  It also severely hinders a teachers ability to help students who are far below grade level improve in their prerequisites, as it is nigh impossible to truly support students across a large range of performance levels.

[4] As a teacher, and as a parent, I know this to be true.  My son continues to believe he is not good at math simply because he was not as successful as other students in elementary school completing his “Fifty in a Minute” worksheets.  I’ve worked with my son for over four years to help him overcome his negative self-talk about his abilities with mathematics.

[5]  The data series labeled “Post CST Assessment” served as the spring end of semester assessment.  It occurred about four months into the second semester, after a lengthy preparation period for the algebra 1 CST.  The four-month time period is nearly identical to the time between taking the diagnostic test at the outset of the course and the end of the first semester.

[6]  I use the same grading scale in all of my courses, which so far include math lab, algebra 1, and AP Calculus AB; I keep student scores as raw percentage scores in my grade book with specific category weights for assignments, quizzes, tests, and the final exam; additionally, if a student passes the final, yet up to that point was failing, I will not fail them – and depending upon the course and their score, if they are failing prior to the final, they may receive the letter grade that corresponds to their final exam score as their final course grade.  At the end of a semester, I consider all the data collected spanning assessments and assignments, and assign a final grade, often moving a student up to the next highest letter grade, + or – included, if they are close to a cut score.  I find the entire area of assessment scoring fascinating. I believe it is nigh impossible to create a statistically valid, normed assessment that properly reflects levels of content understanding, especially if constrained to a traditional 10% bin size for grades A through D.  I do not believe in curving assessments either, as the possibility to skew scores is too high, especially when considering their alignment with levels of understanding such as far below basic, below basic, basic, proficient, and advanced.

[7] I am unable to support the short-term memory versus understanding hypothesis just yet, as I have not scored the assessments based on the correctness of student work.  The data in the graph only show percent correct answers as I gave students the benefit of the doubt when assigning grades for the final exam.  I plan to re-score the assessment based on student work shortly.  At that time, I will speak further to my short-term memory hypothesis.

[8]  As I mentioned in my December, 2012 post, I do not believe student scores were biased much, if at all, by my use of the same diagnostic test each time.  The time period between uses were significant at four months for the first three times, and the fourth time the issue is moot, as I provided students with the answers to each problem the day before the spring final exam for the express purpose of assessing how much short-term memory played in their performance.

[9] It was at this point that my highest achieving student remarked, “Hey, isn’t that one of the questions on a test we took before?”  This goes to show that the third time is the charm, but not the fourth!

[10] I do change my mind if I am convinced it is in the best interest of student learning.  This was not one of those times.

Advertisements

About Dave aka Mr. Math Teacher

Secondary math teacher teaching math intervention, algebra 1, honors precalculus, and AP Calculus AB. I spent 25 years in high tech in engineering, marketing, sales and business development roles in the satellite communications, GPS, semiconductor, and wireless industries. I am awed by the potential in our nation's youth and I hope to instill in them the passion to improve our world at local, state, national, and global levels.
This entry was posted in Assessments, Curriculum, Mathematical Proficiency, Pedagogy and tagged , , , , , . Bookmark the permalink.

7 Responses to Measures of Math Lab Learning Outcomes

  1. Dave says:

    calculators allowed ?
    My brain is too warm to assimilate everything you wrote but you are very comprehensive. The key is not just arithmetic but ” mathematical literacy “, which to me is comfort and understanding of numbers and computational shapes and their relationships to each other; also taking them apart and putting them back together. Like your guvner said I pick it up and I put it down. Many of your problems show this. Maybe more involved problems that have more than one solution would be good as openers. Enuf for now

    Like

    • Calculators were not allowed. I like the NRC’s term, mathematical proficiency, which they defined as having five strands: conceptual understanding, procedural fluency, strategic competence, adaptive reasoning, and productive disposition. Each of these interact with the others impacting one’s overall proficiency with mathematics.

      Like

      • Dave says:

        I like those strands; I’d also love to see, for a particular topic, examples of successful applications of those strands.

        Like

  2. Pingback: Short-term Memory vs. Understanding | Reflections of a Second-career Math Teacher

  3. Pingback: Two Math Teachers Talk | educationrealist

  4. Pingback: Earning One’s Stripes As a Teacher | Reflections of a Second-career Math Teacher

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s