Authors: Kyle M. Whitcomb and Chandralekha Singh
Institution: Department of Physics and Astronomy, University of Pittsburgh, USA
Journal: European Journal of Physics, 41 065701 (2020)
Most university students enrolled in STEM subjects are all too familiar with the idea that their tough introductory science courses are designed to filter out the weaker students through an unforgiving process of survival of the smartest. The working assumption behind such “weed-out” courses is that students’ level of achievement in intro classes provides a practical glimpse of their future achievement in later courses within that subject track. “If you can’t make it here, you aren’t cut out for this subject,” the proverbial weeding-out professor says. There’s a big problem with this logic, however, as the authors for today’s research reveal: the notion that introductory science course grades can prophecy future academic achievement in a science major may be completely bunk.
In today’s paper, a recent study out of the University of Pittsburgh, authors Whitcomb and Singh reveal that students’ graded achievement in intro physics classes does not reliably predict graded achievement in advanced physics classes later down the line. That’s right, the feedback students receive in their Physics 101 and 102 courses is a bad predictor of their eventual success in physics. To make matters worse, the authors found that women physics students tended to underperform in their intro physics courses, so inaccurate feedback from these first courses may be disproportionately convincing women students early on that they don’t belong in the physics department.
What the authors did and found
The study tracked 15 years of high school GPAs, mathematics grades, and physics grades at a single university for students who at some point declared a physics major and stuck with that major at least through the second year of the physics program (i.e., through Modern Physics). For their analysis, the authors grouped courses into four broad categories: introductory physics (Physics 1 and 2), advanced physics (all the remaining physics courses after intro physics), introductory mathematics (Calculus 1 and 2), and advanced mathematics (Calculus 3, Linear Algebra, and Differential Equations).
Further splitting up their data according to gender (and comparing via Cohen’s d), Whitcomb and Singh found that introductory physics courses were the only ones that showed a reliable pattern of gender difference in graded performance. Specifically, in line with the findings of many physics education researchers before, men consistently outperformed their female peers by a narrow margin. Yet, in advanced physics and intro/advanced mathematics, there was no consistent pattern of gender difference. What’s worse, though, is that the women students had come into intro physics courses with slightly higher high school GPAs than their male peers. “[E]ven though men only earn higher grades in introductory physics with a small effect size,” the authors explain, “that small effect size is slightly larger in magnitude and opposite in sign to the effect size of women’s higher average high school GPA.” Already, this suggests that something anomalous is happening in intro physics courses.
After establishing the gender patterns across the surveyed courses, the authors then used a composite statistical method called structural equation modeling (which combines the strategies of confirmatory factor analysis and path analysis) to evaluate the extent to which students’ grades in intro physics and mathematics predicted grades in advanced physics and mathematics. No statistically significant regression paths were found from intro physics to advanced physics, regardless of how the model was tweaked (see Figure 1). That is, the grades that both the men and women physics majors got in their first year of physics did not predict their later physics grades. Instead, advanced physics grades were found to be predicted by mathematics courses. Indeed, to best predict later success in advanced physics courses, Whitcomb and Singh show we should look outside of the physics department altogether.
Figure 1: Three of the structural equation models generated by Whitcomb and Singh (respectively, Figures 3, 4, and 5). In (a), the the advanced physics and advanced mathematics factors were allowed to covary. In (b), the model was tweaked to allow the advanced mathematics factor to predict the advanced physics factor. In (c), the advanced physics factor was split up “according to the typical time-order in which students take the courses.” Note that none of the models show a significant regression path from intro physics to advanced physics. “All drawn paths are significant to the p < 0.001 level except the one[s] denoted with a superscript ∗, which [are] significant to the p < 0.01 level. All missing paths are not statistically significant, with p > 0.05.”
In combining the structural equation modeling results with the gender patterns from before, there emerges a deeply troubling result: students in their first year of physics courses, especially women, will receive inaccurate feedback about their potential to succeed in physics, all at a time when they are making consequential decisions about whether physics (and STEM in general) is somewhere they belong.
Some caveats
The authors point out that course grades are a somewhat crude measure for examining students’ progression through a major, which doesn’t account for many of the other factors that contribute to a student’s sense of self-efficacy. Still, studies have shown that there is a measurable feedback loop between students’ grades and their self-efficacy. The good grades that students get boost their confidence, which in turn can boost their graded performance. Bad grades can send students into a spiral in the opposite direction.
We also need to acknowledge that, in only tracking those students that continued on in their physics major, there is a survivorship bias in this data toward physics majors that did well enough in intro physics to continue onward. Unfortunately, this range restriction is practically unavoidable in this case, since the students that dropped out after the first year of physics by definition didn’t have advanced physics grades to track. Still, for the physics students that drop out of physics after the intro courses (both declared majors and also non-majors who might have declared later on), we cannot say if their earlier grades would have predicted any hypothetical later academic performance in physics.
So, why is this happening?
Overall, Whitcomb and Singh use their paper to reveal problems with university physics curricula more than they explain why the problems arise. Nonetheless, the authors offer one plausible explanation for why intro physics grades do not predict advanced physics grades: there’s a big difference in the competencies valued in the respective course categories. As the authors note, advanced physics courses “essentially test student facility with mathematical procedures as opposed to their conceptual understanding which is typically the focus in introductory physics courses.” Basically, the authors point out that there is a fundamental dissimilarity in what these courses are grading. Advanced physics tend to reward brute mathematical ability while intro physics demands students engage at the conceptual level. If true, this hypothesis would also account for why advanced mathematics grades are a better predictor for advanced physics achievement. Due to the emphasis of advanced physics courses on mathematical procedures, students in upper-division physics are often able to remain successful by virtue of their mathematical abilities, even when their conceptual understanding is lacking.
From this perspective, it’s almost obvious that success at the concept-focused intro level does not predict success at the mathematics-heavy advanced level. Success in the former category entails being competent in a different way than success in the latter. Ironically, accepting this line of reasoning would mean that we have to admit that the longstanding efforts of the physics education community around emphasizing “conceptual physics” (at the introductory level, in particular) have manifested a somewhat incoherent curriculum for physics majors at the transition from intro level to advanced. Spurred on by decades of physics education research that has disproportionately focused on intro physics classes, there has ultimately been a pedagogical shift in only the first year of a physics major’s course catalog. Undoubtedly, the hard-fought push toward ‘conceptual-izing’ introductory physics courses has by many accounts resulted in a much improved first-year physics learning environment, but until advanced courses see similar transformations, many physics majors will nonetheless be facing a discontinuity within their degree.
With regards to gender difference found at the intro level, Whitcomb and Singh point to the explanation that the large-enrollment format of intro courses is especially hard on women students. “Due to societal stereotypes and biases associated with physics,” the authors say, “women may have a lower sense of belonging and self-efficacy in those types of impersonal, non-equitable, and non-inclusive learning environments,” a factor which can have negative implications for their learning of physics.
What should we do about it?
There have long been calls from physics education researchers to stop the pernicious practice filtering out students in our physics courses. It bears repeating, however: today’s authors should convince the last remaining proponents of weed-out courses that there’s a significant crack in their physics crystal balls. Still, for those educators who are not actively trying to weed-out students as much as they are aiming to cover an extensive set of physics content, a recent PERbites summary covered some strategies for how to escape the “content coverage” narrative in physics teaching.
Whitcomb and Singh also emphasize the increasingly-apparent need to make large-enrollment physics courses more inclusive environments for all demographics of students. Their findings add to the chorus of recent research that our physics courses consistently do more harm than good for women students’ self-efficacy for the subject. Working toward this point means developing and implementing strategies for boosting students’ self-efficacy, growth mindset, and sense of belonging in physics.
Finally, the findings from today’s paper have broader implications for physics departments. A physics degree can and should entail a consistent, predictable progression from its introductory courses through its advanced courses. While it may be tempting to lay the blame for current curricula inconsistencies on overly concept-focused intro courses, the onus is almost surely on the advanced physics courses to catch up in valuing physics understanding rather than mathematical pattern matching. Transforming advanced physics courses in this way would better align them with the intro courses that precede them and create a more coherent course sequence through which physics majors could proceed.
When these institutional changes are combined with increased efforts to make intro physics courses more inclusive environments, we have a chance to stop our introductory science courses from sending inaccurate feedback to students soon after they arrive. Then, we just might begin to encourage our capable students from all demographics to avoid the broken future telling of weed-out courses and envision for themselves a physics future they can inhabit.
Figures used under CC BY 4.0. Header image by Elias Euler.
Elias Euler is a postdoctoral fellow in physics education research at Lund University, located in the south-west of Sweden. His research interests revolve around how students and teachers make use of digital technologies to teach and learn physics.