A Slice of PISA

The 2009 PISA results are out and the US finds itself scoring near the middle of the pack for all countries in the OECD (Organization for Economic Cooperation and Development).  For those of you unfamiliar with PISA (Program for International Student Assessment), it is a standardized test administered to approximately 65 countries every 3 years focusing on reading, mathematics, and science.  For 2009, the 65 countries include 34 OECD countries and 31 non-OECD countries.  PISA was designed with a top score of 1000, a mean of 500 and standard deviation of 100.

There have been four PISA testing cycles since its inception in 2000, each covering reading, mathematical and scientific literacy, where students are tested “[not only] in terms of mastery of school curriculum, but in terms of important knowledge and skills needed in adult life,” as determined by OECD member countries.  Imagine the challenge in getting 34 (or 65) countries to agree on standardized test structure, content, wording, and etcetera!

The US Department of Education publishes a summary of the OECD data for each cycle.  Reports for all four earlier cycles are available from the National Center for Education Statistics (NCES).  The 2009 report, released earlier this month, details how the US fared.  Highlights of the 2009 report are also available, as are supplemental tables.

US student performance in mathematics on PISA 2009 at 487 falls below the OECD average of 496.  This places us #25 out of 34 OECD countries, and #31 out of 65 OECD & non-OECD countries, just above Latvia #32 and Lithuania #33, not exactly the position of a world leader.

As shown below, only 10% of US students scored in the top two levels of the PISA performance scale.

US students performed on par with the “average” OECD country in math, which is nothing to write home about.  Worse yet, of the 34 OECD countries included in the 2009 report, the average US math score was only above 9 other OECD countries.

Further review of the data showed no appreciable long-term changes in PISA 2009 math results for US students, on average.  However, long-term increases were clear at the lower end of score percentiles for those who scored below the 50th percentile.

Surprisingly, unlike reports for PISA 2003, there were no breakdowns by subgroups in PISA 2009.  In the 2003 data below, it was clear the achievement gap existed between race / ethnicity, as reported by the US Department of Education.

There was also a clear difference in scores based on the math class a student was taking at the time of the test, which for 15 year olds, would likely be the 9th or 10th grade.  There is data that show the distribution of test takers by grade for those interested.

Given the score disparity in PISA 2003 by race/ethnicity, which is likely present in PISA 2009, how can we close the achievement gap in the US?  How can we improve US students’ performance on PISA? And on NAEP (National Assessment of Educational Progress)?  And other standardized tests (SAT, ACT, etc)?  And on math class attainment in high school (e.g., Algebra 1, Geometry, Algebra 2, Pre-calculus, Calculus)?

To date, a multitude of efforts, projects, initiatives, and studies have not provided a clear-cut answer.  No Child Left Behind (NCLB) held out high hopes, but has not helped.  If anything, the achievement gap is one of the most confounding issues facing our educational system today.  Read the McKinsey Report on the Achievement Gap for in-depth analysis of this challenge.

So what do our PISA 2009 scores say about US education?  Is our ranking and score level distribution more a function of varying US student socioeconomic status (SES)?  Student motivation?  Teacher effectiveness?  Teachers unions?  Curriculum?  Pedagogy?  Parental involvement?  Disparity in school funding?  Other?

Everyone has their pet reason, and it is likely a combination of all of this and more, with the greatest impact outside the classroom from SES and parental involvement (highly correlated) and in the classroom from student motivation and teacher effectiveness (also correlated).

Is it also within the realm of possibilities that, as a country with a steady influx of immigrants fueling our growth, we serve as a “portfolio of students with mixed SES” representative of the “average” OECD country?

And please let me know when you have the answers to my questions.  Of course, after you let Arne Duncan know first.


revised 12/24/10 @ 11:00PM PST

About Dave aka Mr. Math Teacher

Independent consultant and junior college adjunct instructor. Former secondary math teacher who taught math intervention, algebra 1, geometry, accelerated algebra 2, precalculus, honors precalculus, AP Calculus AB, and AP Statistics. Prior to teaching, I spent 25 years in high tech in engineering, marketing, sales and business development roles in the satellite communications, GPS, semiconductor, and wireless industries. I am awed by the potential in our nation's youth and I hope to instill in them the passion to improve our world at local, state, national, and global levels.
This entry was posted in Political and tagged , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s