Authors: Eric W. Burkholder and Carl E. Wieman
First author’s institution: Stanford University
Journal: Physical Review Physics Education Research 15, 020117 (2019)
Each year, thousands of high school students in the United States take Advanced Placement (AP) exams in physics in order to earn college credit and test out of introductory college courses. While each university is allowed to determine which scores qualify for credit, a student typically needs to score at least a 3 (out of 5) to earn college credit. Of the students who took the algebra-based physics 1 AP exam in 2019, 44.6% earned at least a 3 while 71.1% of students who took the calculus-based AP exam in 2019 earned at least a 3.
Despite the wide use of AP exam scores for awarding college credit, there have been few, if any, studies investigating what these AP physics exams are actually measuring or how taking an AP physics exam affects performance in subsequent physics courses. Since AP physics courses are supposed to be equivalent to an introductory university physics course and the course topics are very similar, students who do well on the AP physics exams should also do well in a university physics course and do well on conceptual inventories. Today’s paper examines those claims and finds that AP physics may not be a suitable replacement for university physics courses.
To investigate the effect of taking AP physics on subsequent physics performance, the researchers studied 430 students enrolled in a calculus-based introductory mechanics course (a typical physics 1 course) at a highly-selective university. The researchers point out that their sample may not be representative of typical AP physics students and hence caution about generalizing the results. During the first week of class, the students took the Force and Motion Conceptual Evaluation (FMCE) for participation credit and then completed it again during the last week of the semester. While taking the FMCE at the start of the semester, the students were asked to what type of physics classes, if any, they had previously taken and if they had taken an AP physics course, what their exam score was. Of the students who reported their prior physics experiences, 45 had no prior physics experience, 59 had taken high school physics, 192 had taken an AP physics course, and 104 had taken a non-AP algebra based college-level physics course (such as IB physics). For those reporting their AP scores, the average score was 4. The researchers also collected math SAT scores for each student from university records and found the average score was 730. The researchers then used non-parametric tests (because their data was not normally distributed) to investigate possible differences in FMCE scores and multiple linear regression to model FMCE scores as a function of previous physics classes taken and SAT math scores.
When looking at the scores on the FMCE given at the start of the semester (FMCE pre-test), the researchers found that those with no prior physics experience tended to do poorly on the FMCE while students with some prior physics experience showed a wide spread in their scores (Figure 1 left). Unsurprisingly, those who had taken any prior previous physics course outperformed students who had not taken a physics course before by 26 points in aggregate. However, the type of prior physics didn’t matter. Students who took AP physics did not uniformly outperform students who took any other type of physics class previously (Figure 1 right).
Thinking in terms of effect sizes, the researchers found that students who took AP physics tended to perform 0.38 standard deviations better on the FMCE pre-test than students with other prior physics experiences or no physics at all. In addition, students who had any prior physics experience did 0.51 standard deviations better than students with no prior physics at all. Yet, when the researchers included SAT math scores in their model, the effect size between students who had taken a previous physics class and those who had no physics experience dropped to 0.40 while the effect size for SAT math scores were 0.46. These effect sizes suggest a math test was a better predictor of a student’s physics conceptual inventory score than whether they had previously taken a physics course!
To further study how taking AP physics may impact future physics performance, the researchers looked at three main topics on the FMCE (as identified by Ramlo). They found that a student’s AP physics score was correlated with items about energy, graphical representations, and force/Newton’s Second Law, but not with items about Newton’s Third Law. Interestingly, the correlations disappeared once the students’ SAT math scores were taken into account. Additionally, the researchers found that the AP exam scores had a small correlation with physics final exam scores but that correlation also disappeared once SAT math scores were controlled for.
Despite using a non-representative population, some of the researcher’s findings were similar to previous work. For example, one study found that taking any type of prior physics resulted in better performance on the FMCE pre-test than having no prior physics experience. However, that study and another found that high school physics experience was not a significant predictor of conceptual learning once attitudes toward physics and math preparation were taken into account. On the other hand, some studies have found that the quality of the high school physics instruction is an important factor for predicting college physics performance.
The authors of today’s paper suggest that one reason taking AP physics may not be correlated with FMCE pre-test scores after controlling for SAT math scores is that students may only be recalling formulas rather than understanding concepts, which would explain why students were able to solve energy and Newton’s Second Law questions on the FMCE pre-test but were not successful with Newton’s Third Law questions which are more conceptual in nature. Further, the authors had added 8 basic questions from the Test of Understanding Vectors to the FMCE pre-test and found that the most common score was 0/8 and 50% of those students had taken a college level physics course before (Figure 2). If students were understanding concepts rather than just applying physics formulas, one would expect better performance on basic vector questions given their importance in mechanics concepts.
While the results may not be generalizable due to the students studied, there are a few takeaways that deserve further investigation. First, today’s study suggests that taking any physics before enrolling in university introductory physics is better than having no prior physics experience and that taking AP physics does not appear to result in “better” performance than simply taking normal high-school physics. In addition, students who took any prior physics tended to outscore those who had no prior physics experience on the FMCE post-test as well. Second, SAT math scores seem to be a better predictor of FMCE pre-test scores than whether the student took AP physics or even any prior physics. Taking AP physics also did not appear to have any benefit on the final exam in the university physics course once SAT math scores were taken into account. These results shouldn’t be taken to mean that AP physics do not have an impact, but that their impact may be less than hoped. Indeed, students with any prior physics experience still outperformed students taking physics for the first time on the FMCE post-test. While more representative data from typical students taking AP physics is needed, these initial results suggest that doing well on the AP physics exams may not be a substitute for taking an actual university introductory physics class.
I am a postdoc in education data science at the University of Michigan and the founder of PERbites. I’m interested in applying data science techniques to analyze educational datasets and improve higher education for all students