The Class Affects Observed Teacher Ability

Some teachers excel at teaching what we might call "lower end" students.  That doesn't mean that they don't also excel at teaching higher end students, it's just that they're good at both ends of the spectrum.  Almost any teacher can teach a high end class.

Even teachers who are good at teaching lower classes, though, may not look so good during a classroom observation.  It's reasonable to assume that the classroom environment in those lower end classes will not match the "ideal" classroom environment (which would be akin to an AP Calculus class environment) and as a result the teacher would not be evaluated as highly as his or her results would merit:

Observations of teachers—usually the most prominent component of teacher-evaluation systems—can carry significant sources of bias, potentially penalizing English/language arts teachers of lower-achieving students, concludes a recent research study...

The analysis provides more evidence that, despite the widespread concern about test-score-based ratings of teachers, observations of teachers are just as susceptible to error. It deepens earlier findings by looking at the topic at a more granular level, and by showing that the findings are consistent across several analytical samples...

The researchers found that ELA teachers of higher-achieving students were more likely to get higher scores on the "classroom environment" domain of the Framework for Teaching, which considers teachers' classroom management and their ability to create a respectful learning environment, among other goals. Overall, an English teacher with a class whose incoming achievement was a standard deviation higher would get his or her score pushed up by about a third of a standard deviation...

Now here's the interesting thing. The bias did not show up on all of the FFT domains. Teachers' ratings on things directly related to instructional technique—whether they ask students probing questions and use assessments for instruction—weren't related to students' prior achievement. And math teachers generally didn't seem to get the same boost from having better-performing students. The authors postulate that math instruction, by its very nature, tends to be more direct and to use fewer groups, conversations, and other techniques that rise and fall on student interactions.

Read the whole thing for details and policy suggestions.