Dashboard Delusions – The ED’s Ineffective Measure of Effectiveness

Following President Obama’s recent declaration that by the year 2020 the U.S. will have the largest proportion of college graduates in the world, the U.S. Department of Education (ED) launched a website called the United States Education Dashboard.

From the United States Education Dashboard
Data tell us where we are and how far we need to go. Similar to the dashboard on a car, the United States Education Dashboard shows what the U.S. Department of Education considers to be important indicators of how the nation, and each state, is progressing on the outcomes necessary to achieve the national goal of once again having the highest proportion of college graduates in the world.

The Dashboard is meant to spur and inform conversations about how to improve educational results – conversations among families, teachers, administrators, policy-makers, the business community, and other interested stakeholders.

Not All Dashboards Created Equally

The ED colloquially refers to the website as the Dashboard. Not all dashboards are created equally though, even if well intended.

Do not get me wrong.  Dashboards can be extremely informative.  Honda did a great job with my Accord and Odyssey.  And many companies manage their businesses quite effectively using a variety of dashboards replete with data from finance, sales, operations, engineering, marketing, etc.  However, the data displayed by a dashboard must be a valid indicator before it leads to false conclusions and/or drives well-intended, but misdirected, actions.*

Specifically, the ED’s Dashboard indirectly labels me as ineffective, a soon-to-be credentialed secondary mathematics teacher with the designation “highly qualified” per No Child Left Behind (NCLB), and directly counts me as someone without deep knowledge of the subject I will teach, mathematics.  The ED also implies that future students of mine will receive an inequitable education.  While I hope neither of those interpretations are what the ED intended, it surely places me, and many STEM professionals like myself who are transitioning to teaching as a second career, into a second class status.  One that I do not appreciate.

Why do I make such a bold assertion?  Well, it does not take much of a leap of faith to arrive where I landed after reading the Dashboard and its source material.

To wit, one section of the dashboard describes data for “Teachers and Leaders” as follows.

Goal: To improve preparation, recruitment, development, evaluation, and rewarding of effective teachers and principals.

How the Dashboard Reports on Progress:

  • Is student learning a factor in teacher and principal evaluations?
  • Do high school teachers have the content knowledge needed to teach their specific subject?

To provide an indication of how well the US is progressing on these measures, the ED collects and compiles self-reported data from teachers periodically for inclusion in the Dashboard.  The latest national results for teacher content knowledge, defined by the ED as a teacher having a major in their main assignment area (e.g. English, U.S. History, mathematics, physics, biology, etc), are shown below.

As the figure shows 81.1 percent of public high school teachers in the US majored in their main assignment area, which fell from the last time it was measured at 83.6%.

The ED also breaks down the data for this question by state, as shown below.  As you can see, California ranks in the lowest third of states at 78.4 percent.  I wonder how many of those in the 21.6 percent classified as not having a major in their main assignment have a degree, or other career experience, that makes them more knowledgable than someone with an undergraduate major in their main assignment?

SOURCE: U.S. Department of Education, National Center for Education Statistics, Schools and Staffing Survey (SASS), “Public School Teacher Questionnaire,” 2003-04 and 2007-08.

Furthermore, it is not clear what constitutes an appropriate level: 50 percent, 68 percent, 90 percent, 95 percent, 99.7 percent, or, idealistically,  100 percent?  For an indicator to have value, one should know what to do when reading it.  Now that this measure has dropped nearly 3 percentage points, what should be done?  And is this really a measure of anything meaningful?  This seems to be what I call “measurement myopia,” where laser-focused attention is paid to a metric without really understanding what it truly represents, or whether it is even valid or reliable.  Our infatuation with data acts as blinders on a horse, keeping us faithfully on a path, even if we do not know where we are heading, or what to do based on specific measurement values, as in this case.

Another part of the Dashboard website explains why this measure is important enough to warrant a spot on the dashboard.

Effective instruction requires that teachers have both strong pedagogical skills and content knowledge in their main assignment area. Deep knowledge of the subject taught, as represented by a college major in the subject, is a useful indicator of teacher effectiveness and equity, particularly at the high school level. Over the next 8 years, 1.7 million new teachers will be hired, and recruiting, preparing, placing, and retaining teachers with relevant content knowledge, especially in high-need fields, will help ensure that all students are taught by effective teachers in all subjects. [emphasis added]

While a teacher having a college major in their main assignment area might be “a useful indicator of teacher effectiveness and equity, it is not the sole measure, and may not even be the best or most effective measure.  It certainly is not the best measure of “deep knowledge of the subject taught,” at the high school level.  Having graduated from an undergraduate electrical engineering program, earning a bachelors of science with distinction and recognition, and from a masters of business administration program concentrating in finance and marketing, not to mention a 25 year career in high tech starting out as an engineer and continuing in highly technical marketing and business development roles in the wireless and GPS industries, I surely have deep knowledge of mathematics, especially applied mathematics.  Electrical engineering (EE) programs consist mostly of applying advanced mathematical knowledge, especially in the field of communications theory; think spread spectrum or frequency modulation, not public relations or employee communications.

Unfortunately, the ED is not the only government agency that narrowly defines content knowledge mastery.  As noted in an earlier post, the California Teaching Commission (CTC), apparently does not deem an engineering degree, of any sort, as a marker of mathematical subject matter competency.  I wonder what most of Silicon Valley and the rest of the world’s engineers think about that?  As a card-carrying member of the Institute of Electrical and Electronics Engineers (IEEE), I might bring this issue up for their consideration.  I believe they do not want to see their members cast as ineffective, or lacking a deep knowledge of mathematics, especially that taught at the high school level.

Dashboard Source Data

The data for the specific dashboard measure above relies on responses to the following question by teachers completing the “Public School Teacher Questionnaireissued by the U.S. Department of Education.  While the questionnaire was approved by the US Department of Education, American Federation of Teachers (AFT), National Association of Secondary School Principals (NASSP), and National Education Association (NEA), as well as others, the nuanced interpretation which I describe above clearly was not apparent to them, otherwise, they would not have approved it.  I believe they all acted in good faith in reviewing, and approving the questionnaire, and likely scrubbed the specific question below to make sure it faithfully reflected current ED policy for classifying a teacher as “effective” or not.  However, now sure seems like a good time for the ED to reassess the classification policy.

Table 2 follows.

Note that table 2 lists an engineering degree under “Natural Sciences”  so it, at least, has some recognition by ED.  However, confining engineering, especially electrical engineering, to natural science does degree holders, as well as secondary schools, districts, and students, a disservice.  The educational field seems overly restrictive in mapping degrees to content knowledge.  Is it too insular to see this?

Let’s Change How We Define an Effective Teacher

While I applaud President Obama’s 2020 Education Goal, I believe it is important that the ED re-evaluate how they define certain measures.  With the recent push to increase the STEM presence in secondary education, it makes no sense to seek out those with engineering and similar degrees to teach mathematics, as an example, but classify them as lacking effectiveness in their subject area, or worse yet, furthering inequitable education.

Let’s not send mixed messages like this to an important source for future teachers.  While I am still peeved that I will be classified as an “ineffective” teacher using the ED’s definition, I know that I am, and will continue to be, a highly effective teacher, from a content knowledge standpoint.  I still have a few years to go to become “highly qualified” from a pedagogical standpoint, as I clearly claim in an earlier post:  “Highly Qualified” Interns – a Mendacious Misnomer.

* Too often the data from surveys have questionable validity, especially when they are the results of questionnaires, which can be unintentionally biased based on question wording.  Crafting survey questions is an art and a science, especially if the questions are to be used for multiyear tracking studies, also known as longitudinal studies.  A poorly crafted question can invalidate years of comparative data.

About Dave aka Mr. Math Teacher

Secondary math teacher teaching math intervention, algebra 1, honors precalculus, and AP Calculus AB. I spent 25 years in high tech in engineering, marketing, sales and business development roles in the satellite communications, GPS, semiconductor, and wireless industries. I am awed by the potential in our nation's youth and I hope to instill in them the passion to improve our world at local, state, national, and global levels.
This entry was posted in Political and tagged , , , , , , . Bookmark the permalink.

3 Responses to Dashboard Delusions – The ED’s Ineffective Measure of Effectiveness

  1. Pingback: Tweets that mention Dashboard Delusions – The ED’s Ineffective Measure of Effectiveness | Reflections of a Math Teacher Candidate -- Topsy.com

  2. Cal says:

    What gets me is that a math major has never been either necessary or even sufficient to become a math teacher in California. You have to pass a test.

    So why isn’t passing the test proof of high qualifications?

    Many school districts won’t hire you if you don’t have a math major, either.


  3. Pingback: A New Teacher Says He Is Not “Highly Qualified” « Diane Ravitch's blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s