Lights! Camera! Doh!

While I believe periodic, limited, and controlled digital video recording of a teacher leading instruction offers tremendous opportunities for professional improvement via self-reflection by the teacher, and suggestions from a teacher’s coach, it should not be legal to use it as a basis for high-stakes decision-making, performance evaluations (except as substantiation to justify in-person observation ratings), punitive or disciplinary use, or anything aside from helping the teacher improve in their profession, without the threat of penalties, period.

Additionally, without the entire classroom in each frame, important contexts are likely missing that are needed to factor into any fair judgment or interpretation of the teacher’s behavior on video, except when used for non-punitive use such as coaching or self-reflection, where the observer makes notes to supplement the recording, adding the missing context in many situations.

Furthermore, access to each video file should be restricted by law to the teacher, coach, principal, superintendent and their limited designees, unless otherwise agreed to by the teacher; otherwise, it could end up in the public domain, or in malicious hands, where it could be edited, misinterpreted, or used to embarrass the teacher.  As the LA Times has shown, as well as WikiLeaks, digitally recorded and stored information can be accessed too easily and twisted to suit a specific entity’s purpose, well-intentioned or not.

Unfortunately, there is just too much potential for abuse, misinterpretation, malicious intentions, reprisals, and etcetera.  Many considering this approach overlook the significant pressure associated with the event will impact the teacher and his or her instruction since few people, aside from actors, are comfortable knowing they are being observed and/or recorded; it is simply human nature to feel nervous, which will impact one’s actions, especially the newer the teacher or the more concerned they may feel beforehand and during the video recorded instruction.  This is commonly known as “performance anxiety.”

Video recordings are also ripe for several biases to include observational bias, confirmation bias, measurement bias, assumption bias, and others, which is why they are great for collaboration between two parties present during the recording since they can discuss the potential biases present before they become entrenched and interpreted as “fact.”

At the same time, it is clear that there is a strong bias present in the current teacher evaluation system, otherwise we would not hear statistics that most teachers are rated as effective, typically due to a limited survey instrument with a Pass / Fail scale.  Hence, significant effort is being spent in this area of late.  A quick Google search shows states, cities, and districts using comprehensive instruments that seem difficult to misrepresent measurement-wise, if we trust in the principal to make fair assessments; at the same time, video can be helpful to review if any claims of bias are made, whether for over-rating or underrating, by any principal, once probable cause exists.  See North Carolina, San Diego County, and Cincinnati for examples, as well as organizations such as The New Teacher Project, who are developing guidelines for effective teacher evaluation systems.

I have personal experience with a teacher evaluation system like that used by San Diego County.  As a teacher candidate in California, I am subject to the California Standards for the Teaching Profession (CSTP), revised in 2009 by leading educational experts serving on an advisory panel to the California Commission on Teaching Credentialing (CTC).   In this use, I believe they are effective for teacher candidates to develop awareness of teaching’s complexity and the breadth of domain expertise required, as well as to monitor advancement along these standards throughout their student teaching experience.  For those graduating from educational programs using this instrument and process, it may even be acceptable to them to continue its use since they are comfortable with it, so they may monitor their continued development as they advance in their career.

At the same time, the principles I mention at the outset of this post still apply.  Video recordings of teacher instruction should only be used for non-punitive purposes, otherwise, they will be feared and accelerate the downward spiral of our nation’s educational system if teachers leave due to fears over a “big brother” abuse of digital video, or worse yet, seeing themselves on YouTube making an unfortunate gaffe or mistake, which is all too common when on your feet in front of 150+ adolescents over a six plus hour length of time, five days a week, ten months of the year.

Think about it.  Would you want video of you while working at your job available for anyone to see, anytime they wanted, with a stream of comments posted by anyone with an internet connection?  I did not think so.

Teacher Ratings Get New Look, Pushed by a Rich Watcher


Travis Dove for The New York Times

Lindsey Cozat, a technology teacher at Croft Community School in Charlotte, N.C., set up a classroom recording device.

Published: December 3, 2010

PRINCETON, N.J. — In most American schools, teachers are evaluated by principals or other administrators who drop in for occasional classroom visits and fill out forms to rate their performance.

Travis Dove for The New York Times

The aim is to capture what happens in the classroom of a fifth-grade teacher, Damien Kingsberry, and to evaluate him.

The result? More than 9 out of 10 teachers get top marks, according to a prominent study last year by the New Teacher Project, a nonprofit group focusing on improving teacher quality.

Now Bill Gates, who in recent years has turned his attention and considerable fortune to improving American education, is investing $335 million through his foundation to overhaul the personnel departments of several big school systems. A big chunk of that money is financing research by dozens of social scientists and thousands of teachers to develop a better system for evaluating classroom instruction.

The effort will have enormous consequences for the movement to hold schools and educators more accountable for student achievement.

Twenty states are overhauling their teacher-evaluation systems, partly to fulfill plans set in motion by a $4 billion federal grant competition, and they are eagerly awaiting the research results.

For teachers, the findings could mean more scrutiny. But they may also provide more specific guidance about what is expected of the teachers in the classroom if new experiments with other measures are adopted — including tests that gauge teachers’ mastery of their subjects, surveys that ask students about the learning environments in their classes and digital videos of teachers’ lessons, scored by experts.

“It’s huge,” said Deborah Loewenberg Ball, dean of the University of Michigan School of Education. “They’re trying to do something nobody’s done before, and do it very quickly.”

The Gates research is by no means the first effort of its kind. Economists have already developed a statistical method called value-added modeling that calculates how much teachers help their students learn, based on changes in test scores from year to year. The method allows districts to rank teachers from best to worst.

Value-added modeling is used in hundreds of districts. But teachers complain that boiling down all they do into a single statistic offers an incomplete picture; they want more measures of their performance taken into account.

The Gates research uses value added as a starting point, but aims to develop other measures that can not only rate teachers but also help educators understand why one is more successful than another.

Researchers and educators involved in the project described it as maddeningly complex in its effort to separate the attributes of good teaching from the idiosyncrasies of individual teachers.

Mr. Gates is tracking the research closely. The use of digital video in particular has caught his attention. In an interview, he cited its potential for evaluating teachers and for helping them learn from talented colleagues.

“Some teachers are extremely good,” Mr. Gates said. “And one of the goals is to say, you know, ‘Let’s go look at those teachers.’ What’s unbelievable is how little the exemplars have been studied. And then saying, ‘O.K., How do you take a math teacher who’s in the third quartile and teach them how to get kids interested — get the kid who’s smart to pay attention, a kid who’s behind to pay attention?’ Teaching a teacher to do that — you have to follow the exemplars.”

The meticulous scoring of videotaped lessons for this project is unfolding on a scale never undertaken in educational research, said Catherine A. McClellan, a director for theEducational Testing Service who is overseeing the process.

By next June, researchers will have about 24,000 videotaped lessons. Because some must be scored using more than one protocol, the research will eventually involve reviewing some 64,000 hours of classroom video. Early next year, Dr. McClellan expects to recruit hundreds of educators and train them to score lessons.

The goal is to help researchers look for possible correlations between certain teaching practices and high student achievement, measured by value-added scores. Thomas J. Kane, a Harvard economist who is leading the research, is scheduled to announce some preliminary results in Washington next Friday. More definitive conclusions are expected in about a year.

The effort has also become a large-scale field trial of using classroom video, to help teachers improve and to evaluate them remotely.

“Video lasts,” Dr. McClellan said, creating possibilities for dialogue among teachers about improving classroom techniques. “Colleagues can watch your video and say, ‘Right here — where you did that — try this next time.’ So the teacher learns a new skill.”

There are advantages for teacher evaluations, too, Dr. Kane said.

With videos, for instance, several professionals, rather than just one principal, could rate the same classroom performance, making ratings less subjective, he said.

“It potentially creates a cottage industry for retired principals, or even expert teachers, to moonlight on weekends scoring classroom observations,” he said.

An Internet-based approach to teacher evaluation could also alleviate some pressures on school districts. New laws in many states, after all, are requiring more frequent observations of teachers.

A new evaluation system in Washington, D.C., for example, requires five observations each year, compared with the previous systems that required one or two at most, and in many cases none at all. Starting next fall, a Tennessee law will require at least four observations a year, rather than one every five years.

In some districts, the increased pace is straining the workload of administrators. Memphis officials realized that under the new rules, their district would need to conduct more than 28,000 classroom observations annually, a task that could overwhelm the city’s school principals.

“This technology can help us face the logistical challenge of being in so many places at the same time,” said John Barker, who leads the district’s research and evaluation office.

The district still intends to have principals visit classrooms, but in January will start a pilot program to use videotaped observations, he said.

Dr. Kane said the foundation hoped more school districts would start using classroom videos, for training and for evaluations, and has worked to keep costs down.

Teachscape, a contractor providing cameras, software, and other services for the research, estimated first-year startup costs of about $1.5 million for a district with 140 schools and 7,000 teachers to buy one camera per school and lease the software to carry out classroom observations using digital video. After that, annual costs would drop to about $800,000, said Mark Atkinson, the chief executive of Teachscape, which is based in San Francisco.

In addition to the cost — which many struggling districts may consider too high — another barrier could be teacher opposition. The Memphis teachers union, an affiliate of the National Education Association, has partnered with the foundation for the project. But Keith Harris, its president, said the use of videotaped observations in evaluations raised troubling questions.

“Whose eyes would see these videos?” Mr. Harris asked. “Who would own them? This seems like an ‘I gotcha’ kind of thing. We think these observations deserve a human being.”

Randi Weingarten, president of the American Federation of Teachers, which has several affiliates participating in the research, also expressed reservations. “Videotaped observations have their role but shouldn’t be used to substitute for in-person observations to evaluate teachers,” Ms. Weingarten said. “It would be hard to justify ratings by outsiders watching videotapes at a remote location who never visited the classroom and couldn’t see for themselves a teacher’s interaction and relationship with students.”

Dr. Kane said doubts may disappear with time. “We’re not naïve,” he said. “We realize that most principals and teachers imagine an in-person visit from a human being when they think of classroom observations. But that could rapidly change. It’s not out of the realm of possibility that millions of classrooms could be using this technology within four or five years.”

A version of this article appeared in print on December 4, 2010, on page A1 of the New York edition.

About Dave aka Mr. Math Teacher

Secondary math teacher teaching math intervention, algebra 1, honors precalculus, and AP Calculus AB. I spent 25 years in high tech in engineering, marketing, sales and business development roles in the satellite communications, GPS, semiconductor, and wireless industries. I am awed by the potential in our nation's youth and I hope to instill in them the passion to improve our world at local, state, national, and global levels.
This entry was posted in Political and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s