The value of subjective and objective evaluations of teacher effectiveness

(Press-News.org) NEW YORK – October 17, 2011 – A study conducted by Columbia Business School's Prof. Jonah Rockoff, Sidney Taurel Associate Professor of Business, Finance and Economics, and Cecilia Speroni, a doctoral student at Teachers College, set to estimate whether subjective evaluations of teacher effectiveness have predictive power for the achievement gains made by teachers' future students. The study, which was recently published in Labour Economics, found that subjective evaluations are comparable with and complementary to objective measures of teacher effectiveness taken from a teacher's first year in the classroom. The study's analysis drew on data on students and teachers in the public schools of New York City – specifically teachers of grades 3 to 8 in the school years 2003-04 through 2007-08, and their students' behavior, demographics, and achievement test scores in math and English.

Prof. Rockoff found evidence that teachers who receive better subjective evaluations of teaching ability prior to hire or in their first year of teaching also produce greater gains in achievement, on average, with their future students. Consistent with prior research, the results also support the idea that teachers who produce greater achievement gains in the first year of their careers also produce greater gains, on average, in future years with different students. More importantly, subjective evaluations present significant and meaningful information about a teacher's future success in raising student achievement. This finding is particularly noteworthy, since the variation in subjective evaluations may capture facets of teaching skill that are not captured by the study of standardized tests results.

Using the linked student–teacher data, the researchers objectively evaluated teachers' impacts on student test scores using an empirical Bayes' method. In terms of subjective evaluations, data came from New York City Teaching Fellows (TF), an alternative path to teaching certification taken by about a third of new teachers in New York City. Alongside this data, they also used data from New York City's "New Teacher Induction" program which spanned the school years 2004-2005 through 2006-2007. Under this centrally administered program, a group of trained, full-time mentors worked with new teachers over the course of their first year to improve their teaching skills and submitted ongoing evaluations of teachers' progress in mastering a detailed set of teaching standards. The study then proceeds to examine student achievement in the second year of teachers' careers; the researchers believe this provides a more rigorous test of whether objective and subjective performance metrics provide useful information for decisions such as teacher retention.

Consistent with prior research, first-year value-added estimates were significant predictors of student achievement in the teacher's second year. Furthermore, evaluations by mentors – and in particular variation in evaluations within mentors – bear a substantial positive relationship with student achievement in teachers' second years.

The study suggests that evaluation systems that incorporate subjective measures made by trained professionals and objective job performance data have significant potential to help address the problem of low teacher quality. However, the researchers warn that the application of standards can vary significantly across individuals responsible for making evaluations, and the implementation of any evaluation system should address this issue.

###

About Columbia Business School

Led by Dean Glenn Hubbard, the Russell L. Carson Professor of Finance and Economics, Columbia Business School is at the forefront of management education for a rapidly changing world. The school's cutting–edge curriculum bridges academic theory and practice, equipping students with an entrepreneurial mindset to recognize and capture opportunity in a competitive business environment. Beyond academic rigor and teaching excellence, the school offers programs that are designed to give students practical experience making decisions in real–world environments. The school offers MBA and Executive MBA (EMBA) degrees, as well as non–degree Executive Education programs. For more information, visit www.gsb.columbia.edu.

END