Authors: Eric Burkholder, Lena Blackmon, and Carl Wieman.
First Author’s Institution: Department of Physics, Stanford University, USA.
Journal: Phys. Rev. Phys. Educ. Res. 16, 020134 (2020).
Physics education research has identified differences between experts and novices in how they approach and solve physics problems. For example, in the classic study by Chi et. al, experts (PhD students) took longer than novices (undergraduates who had just completed a semester of mechanics) to classify 24 problems into various categories. Also, the categories that the study participants came up with were dramatically different for the two groups. The experts came up with categories that reflected physics principles such as energy conservation where as the novices came up with categories that reflected surface features such as “inclined plane problems”. This paper by Chi et al is discussed in more detail in a PERBites post. .
One aim of physics education research is to characterize such differences between experts and novices, and then to develop strategies that allow novices to develop more expertise in problem solving. In this study the authors explore the frequency with which students, who are no-longer novices but are not experts either (“transitioning novices”), use various problem solving strategies that they were explicitly taught in class.
Based on prior research on authentic problem solving and how experts solve problems, the authors identified 5 activities that map to different levels of expertise. These strategies are relevant to the material that the participants of this study learned in class and the various activities in the class were designed to help students learn and use these strategies. The strategies are listed below.
Most demanding strategies (make predictions).
Checking limiting values of the angular dependence of an expression to check whether the angular dependence is correct.
Intermediate level strategies (identify relationships).
Checking the units of the expression on either side of an equation and making sure there is agreement.
Identifying Components of vectors
Deciding on solution correctness based on identified vector components of forces and torques that are relevant to solving the problem.
Using definition of torque to evaluate the given solution.
Basic level strategies (identify important factors).
Identifying Functional Relationship
For example: the height to which you can climb on an inclined ladder should depend on the coefficient of friction.
The authors then designed 2 problems that give students the opportunity to engage in these activities. The two problems are named the “ladder” problem and the “shelf” problem. Each problem comes in two variants: “contrast” and “verify”. The contrast variant asks students to pick the correct answer and the verify variant asks students to find errors, if any, in the given expression. One variant of each problem is show in figure 1 below (figure 1 in the paper): the left panel shows the contrast variant and the right panel shows the verify variant.
The research methodology is to inspect the frequency with which students use the 5 strategies while answering these questions.
The students who participated in this study had taken an introductory physics course before taking the class (Physics 41E described below) in which this study was conducted. The introductory course is one that was specifically designed to teach students to solve problems like expert physicists. Prior research on the introductory course had shown that students did get better at some tasks such as planning their solutions to problems, but at the same time they were not able to do other expert like activities such as identifying important elements in a problem and making assumptions. Thus these students have progressed from the complete novice stage but can still make more progress towards being experts. The researchers call this group of students “transitioning novices”.
Physics 41E, the class in which this study was conducted, covers static equilibrium, forces, torques, 1D kinematics, and conservation of energy. The class used problems designed by the authors to include elements of authentic problem solving. These elements include identifying important variables, predicting functional relationship between variables, making predictions about limiting behaviors, and checking whether the solutions made any physical sense. Due to this emphasis in the class it is reasonable to expect that the students will use some of these strategies in the questions given as part of the experiment. This allow us to see how much progress the “transitioning novices” are able to make in their journey towards becoming more expert-like in problem solving.
The experiment was conducted during the last week of classes. The students had 30 minutes to complete the experiment and they were given attendance credit for completing the experiment. 78 of the 95 students in the class participated in the study.
Each student sees one version of each question. This leads to four possibilities. Two verify-first cases, “ladder-verify shelf-contrast” and “shelf-verify ladder-contrast”, and two contrast-first cases, “ladder-contrast shelf-verify” and “shelf-contrast ladder-verify” (in the final analysis the order, verify-first or contrast-first, didn’t matter). Each student sees one of these four possibilities. The authors asked the participants to give answers as free responses in which they explained why they chose a particular answer. They also instructed the students to not derive solutions, and only to check the provided answers and write down their reasons for choosing an answer.
The authors coded each student response to identify the different strategies that the student may have used in a given response. They then tabulated the percentage of students engaged in each of these strategies. They also tabulated whether students got the correct answer or not. Success in the contrast case meant that the student identified the correct expression, and success in the verify case meant that the student correctly identified the errors in the expression.
The use of the most demanding strategy, evaluating limits, is minimal. In the contrast version 0% and 11% of students, in shelf and ladder cases respectively, used this strategy. In the verify version 11% and 27% of students, in shelf and ladder cases respectively, used this strategy. Note that this doesn’t mean that they only used this technique — a given response has multiple strategies in it. It seems that despite repeated and explicit instruction in using this strategy, very few attempted the strategy. The authors reason that the level of cognitive complexity required in applying this strategy could be the main reason for this. The prompt available in the verify version could be the reason that students used this strategy more in the verify version than in the contrast version.
The most used strategies are the intermediate ones. For example in the contrast version, 49% and 32% of students, in shelf and ladder cases respectively, used the identifying components strategy. In the verify version these numbers are 62% and 32%. Similarly, the numbers for the unit analysis strategy are higher than those for the evaluating limits strategy. The fact that students are using these strategies mean they are indeed moving away from novice like problem solving behavior.
The authors note that evaluating limits is more widely used in the ladder problem than in the shelf problem. They reason that this is likely because the ladder problem was harder and other techniques do not work as much as they do in the shelf case.
This study shows that transitioning students were successful in applying intermediary techniques such as identifying relationships between components and analyzing units. These students had difficulty using the more advanced technique of evaluating limiting behaviors, even though the students were taught this method in class. These results are aligned with findings from other studies and suggest that careful scaffolding will be required to enable students to learn problem solving techniques that have high cognitive demands. In addition, when calling on students to use more difficult strategies instructors should ensure that simpler strategies are inadequate for the problem at hand.
This study does have some limitations. The study was conducted at Stanford which is a very exclusive university, and so PER researches should look to conduct such experiments in other institutions to see any variations in the results. It would also be valuable to compare how frequently the students used these strategies for in-class assignments: did they actually learn how to use these methods from in-class activities?
The authors provide some suggestions on how to use the findings from this paper as well as from related prior research. Instructors should teach intermediary strategies first since they are easier for students. They should then build on these strategies to teach more involved strategies such as evaluating limits since such techniques impose more cognitive load. Explicit instruction, with scaffolding that recognizes the varying degrees of cognitive demands, is needed for students to effectively move towards using more expert like problem solving strategies.
Prasanth Nair is a freelance software developer with strong interests in STEM education research, especially Physics Education Research.