This is Interesting

There is an interesting study out of Finnish teachers.

Because the Finns treat their teachers well, admissions to teaching schools are highly competitive, with only about 10% of applicants making it into teaching schools.

Until 1989, in an attempt to maintain gender balance among teachers, it was required that 40% of all admissions be males, and when this requirement was dropped, and the first stage of admissions were based only on high school matriculation exams.

It turns out, that based on student outcome, classes after 1989 produced teachers of lower qualities.

The conclusion is that the reliance on these sorts of tests, which forms the basis of many claims of meritocracy, did not actually result in better performance:

I recently came across a paper by Ursina Schaede and Ville Mankki that contains a fascinating empirical finding with major implications for the way in which we think about meritocracy.

The paper examines the long run effects on students of a change in the manner in which their teachers were selected into a graduate program. Finland is well known for having an extremely effective school system, in part because primary teacher education has been “exclusively taught as a research-oriented, five year masters’ degree at universities” since the 1970s. These programs are in very high demand among applicants, with acceptance rates of about 10 percent. The admissions process has a first stage based largely on scores on a high school matriculation exam, followed by a second stage involving interviews and the evaluation of live teaching. Candidates are ranked again at the end of the process, and those at the top are taken until capacity is filled.

For a number of years, acceptance into the second stage was based on a quota, ensuring that at least 40 percent of students making it through the first stage of evaluation were male. Although this did not place any constraint on second stage outcomes, it turned out that the entering classes (and hence several cohorts of trained teachers) did not differ much in gender composition from those making it through the first stage. The quota was abolished in 1989, leaving first stage outcomes unconstrained. The first post-quota cohort thus graduated in 1994.

The paper examines the causal effects of this change on the long run outcomes for students. Identification is facilitated by variations across municipalities in the age distribution of teachers at the time of quota removal, coupled with mandatory retirement at age 60. This means that students were differentially exposed to the newer post-quota cohorts, which had a different gender composition (fewer males) and a different distribution of scores on the matriculation exam (higher scores on average).

The authors find that students differentially exposed to the quota-constrained cohorts of teachers ended up with better educational attainment and labor force participation at age 25. In other words, removal of the quota led to a decline in student performance. While this finding is interesting in its own right, even more interesting are the mechanisms that the authors rule out, and the one that they eventually accept.

………

What, then, do the authors think is driving their results? They argue that the evidence “is consistent with male quota teachers contributing positive qualities to the school environment that are not sufficiently captured by the selection criterion in absence of the quota.”

It is important to be clear about this, because the finding can be so easily misunderstood. It is not that the quota teachers proved effective because they were male. It is that the distributions of important characteristics (unmeasured by scores) were not identical across male and female applicant pools. The quota was picking up individuals with these characteristics by proxy. It is the characteristics that mattered for students, not the gender of the teacher.

High stakes testing selects for the ability to take high stakes tests, and that is often unrelated to actual performance.

H/T Brad DeLong