Teacher Science Knowledge and Student Science Achievement - meta-analysis

Becker, B.J. & Aloe, A.M. (2008). Teacher Science Knowledge and Student Science Achievement. Paper presented at the annual meeting of the American Educational Research Association, New York, March 2008

In this research the authors ask whether teachers’ knowledge of science is an important predictor of student science achievement. the authors use meta-analysis to examine a set of studies done in the United States since 1960, and find that increased content knowledge has a positive and significant, but small, bivariate relationship with student achievement. However, effects from more complex analyses are essentially nil.

The authors address these questions:
1) Are teachers’ levels of content knowledge in science related to their students’ achievement?
2) Do differences in school level, area of science achievement, and how teacher knowledge is measured relate to the strength of relationship found?
3) For more complex studies, does whether prior student achievement is controlled affect the strength of relationship between teachers’ science content knowledge and student achievement in science?

The authors analyses have revealed, first, that the amount of evidence on the relationship of science teacher knowledge to student outcomes in science is not extensive, and is not of very high quality.  Fewer than 30 studies report results pertinent to this topic. The bulk of those studies have measured teacher knowledge using variables that appear to be, at best, proxy variables for actual levels of knowledge. If these measures indeed have low validity as indices of content knowledge, the correlations that the authors have observed will be lower than the true correlation values (see, e.g., Hunter & Schmidt, 2004, for a discussion of invalidity as an artifact in meta-analysis).

The correlational studies reveal that, on average, teacher knowledge in science has a slight positive correlation with student science achievement. However, there is variation that is not accounted for by any of the explanatory variables examined in the authors analyses. More notably, when other variables are held constant (in the five studies reporting regressions), the relationship disappears.

These results are a bit oversimplified, however.  The authors analyses revealed that a variety of factors relate to the size of the bivariate correlations between measures of teacher content knowledge and student science achievement. Of most interest is that when the authors examine studies where the measures of teacher knowledge and student achievement focus on the same content, two content areas show larger correlations: .32 on average for biology and .18 for physical sciences. Though neither of these values is large, they are larger than the means overall and for any other subsets of effects.

Also worth noting are the conflicting results for numbers of credits in science and counts of courses taken. While the mean r for credit hours in science was a significant .15, the mean for course counts was significantly negative, if trivial, at -.03. These two proxy-like predictors were studied in different kinds of samples, which may have affected the values as well.

Finally there is the issue of the national samples. In the authors analyses of correlations, the results of the national studies, all of which were based on large multi-stage probability samples, showed essentially no relationship of teacher knowledge to student achievement. The same was true for the regression studies, all of which similarly drew on national probability samples. These studies might be considered to provide the authors “best evidence” on the issue at hand, since they allow for inferences to be made to a well defined population. However, some caution is in order due to the nature of the measures used in these studies. All of these studies used broad measures of student science achievement, not measures of specific science content. Similarly all used coarse self-report proxies to represent teachers’ science knowledge, many of which were course counts and indicators of whether a teacher had a major or an advanced degree in science. These may have a less direct relation to the construct of interest than more targeted measure of teacher knowledge, thus attenuating the observed relationship. Last, two of the national surveys (LSAY and NELS) appear both data sets, and appear twice in the set of regression results, because several authors have analyzed these important data sources. Thus their findings, which appear to be weaker than those of local samples, play a large role in the authors conclusions.

Clearly, a multitude of factors aside from teacher subject-matter knowledge have been documented to impact student science achievement. Such things as students’ verbal, spatial and reasoning skills (Piburn, 1993), a diversity of teaching strategies (Bowen, 2000; Johnson, Kahle, & Fargo, 2007; Schroeder et al., 2007) and curricular interventions (Shymansky, Kyle, & Alport, 1983) have been found to impact achievement. Piburn  (1993) reported correlations ranging from about .20 to .45 for ability predictors with school science outcomes, larger than most of the values the authors report.  Bowen (2000) examined a set of studies of cooperative learning activities in high-school and college chemistry courses, and found effects that would average about .18 on the correlation scale. Other influences have been found to have even more sizeable impacts on achievement. For instance, Schroeder and colleagues (2007) examined 61 experiments or quasi-experiments on a wide range of science teaching strategies. They found effects ranging from .14 to roughly .60 on the correlation scale.

We can also compare the authors results to those found in a recent meta-analysis of the importance of subject-matter knowledge in mathematics. Choi, Ahn and Kennedy (2007) examined  results from 16 studies of student math achievement. They found that teachers’ arithmetic knowledge correlated on average only .07 with student arithmetic performance, while results were mixed for algebra achievement. Performance on algebra concept tests showed a correlation of .12 with teacher knowledge, whereas computation in algebra was unrelated to teacher knowledge. As was true for the authors' analyses the correlation of teacher knowledge to student math outcomes varied according to a variety of features of the studies and measures used. However, none of the mean correlations reported by Choi and colleagues averaged above .2, and as was true for the authors results, some mean correlations were significantly negative.

Re: claims that have been made about the importance of subject matter knowledge - In 2002 the report of the Secretary of the U.S. of Education asserted “Rigorous research indicates that verbal ability and content knowledge are the most important attributes of highly qualified teachers” (2002, p. 19, emphasis added).   the authors would argue that teacher knowledge is only one factor among many other more important ones leading to increased science achievement for students. It may be that with more targeted, higher-quality measures of teacher content knowledge, a stronger relationship would be found. This leads to one clear suggestion for future work: Use better, more specific measures of teachers’ science knowledge. However, the existing literature suggests that the relation of teachers’ content knowledge to science achievement is weak, and provides a very poor basis for claiming that science subject-matter knowledge is among the “most important” attributes of highly qualified teachers.