Fascinating post from William Schmidt, Distinguished Professor at Michigan State University and co-director of the Education Policy Center. Schmidt compares the differences between the PISA and TIMMS assessments. In a nutshell, he notes the
Major distinction in what the two tests purport to measure: the TIMSS is focused on formal mathematical knowledge, whereas the PISA emphasizes the application of mathematics in the real world, what they term “mathematics literacy.”
In other words the TIMMS is closer to what many people criticize about assessment. It measures students ability to solve problems in discrete areas with proficiency. This, skeptics note, is not like the real world where problems require intuition and mathematical literacy–“number sense.” I’m glad there are assessments of both and I think we should always try to make our assessments more rigorous–rather than trying to use questions about the rigor of some questions as a rationale for not assessing, say, but it’s hard to underestimate how simple and important what Schmidt observes is. (Or what it may be. See below for a major caveat).
After analyzing the new PISA data, we discovered that the biggest predictor of how well a student did on the PISA test was exposure to formal mathematics. This is a notable finding, to be sure, since the PISA is designed to assess skill in applied rather than formal math. Exposure to applied mathematics has a weaker relationship to mathematics literacy, one with diminishing marginal returns. After a certain point, more work in applying math actually is related to lower levels of mathematics literacy.
In other words what Schmidt finds is that the thing that most enables students to succeed in applied math is the strength of their foundation in “formal math.” Surely they should also get experience in applied “real world” math, but that experience ,must supplement and not supplant the formal foundation. In other words, if you skip the formal and just do applied math you get very little math. And if you try to make math more rigorous by over-weighting the applied thinking you may, in the end, get reduce outcomes.
Why is this so important? Well, there’s a bit of a tendency in US math, you’ve probably noticed, to pit “higher order thinking” or “conceptual thinking” or “number sense” against formal math which, in such situations, is often caricatured as “rote learning” or, at its most pejorative, “drill and kill.” But the two “opposites” are actually not just synergistic but inter-reliant. Everyone loves to admire a shining tower but the building depends on the foundation (and I should note, a foundation ain’t much without a building), so foundation versus superstructure arguments are counter-productive. We should replace them with foundation AND superstructure arguments.
Further there may be something counter-intutitive about where the right balance is. We think we’re winning when we get as much of the abstraction in as we can but in fact we might get more value out of our abstraction if we did a little less of it, a little better informed by the concrete. In a Japanese parable a student of the Zen tea ceremony asks, “Master, why do we do it this way?” To which the master replies: “Do it a thousand times and you will understand.” Wisdom and intuition come from deep formal knowledge. At least that’s one possible interpretation.
Two final points. 1) a big caveat 2) an application note
The caveat. Paul Bruno, who is consistently insightful in looking at research on teaching and learning, notes that the findings are undercut by their reliance on self-report. In other words, the PISA results are correlated to students descriptions of how much formal math they got and that is highly unreliable. So we have a pretty big methodological caveat and have to take this with a grain of salt… not sure how big but certainly not insignificant.
In the meantime, Paul Powell, principal of Troy Prep Middle School and who I am trying to convince to writing a book on math one day, has designed a program that appears to align to Schmidt’s findings. They go “high low.” Lots and lots of fast practice automaticity with basics. Strong foundation in the formal. Supplemented by Olympiad Math type problems several times a week. In other words, they build up constant formal knowledge for 90 or more minutes a day and then spend 20 minutes once a day or maybe every other day doing a problem that requires problem solving and math literacy–specifically where students not only have to solve but have to figure out how to solve and to use multiple tools to solve. The questions are very demanding. The kids get better and better at them as they practice. And their practice is constantly informed by a base of new foundational skills. His students routinely score in the top 1% of schools in New York state in math and last year, even with a FRPL percentage of about 95%, his students passed the state test at two and a half times (74% versus 28%) the state average. The year before that they were among the top 20 schools in New York in their percentage of students not at level 3 but at level 4.