On Error

Say the answer you get is 3 when you’re supposed to get 4. Did you do something wrong? Most of the students I taught in freshman physics lab courses seemed to think so.¹Having now twice started a blog post by gently ribbing first-year physics students, I feel compelled to emphasize that such pitfalls at the beginning of a physics education should be no cause for shame. Undergrads I taught in these lab courses went on to grad school at great universities all over the country and I expect many of them now know a good deal more about physics than I do. I can’t tell you how many times I saw statements in lab reports like “Our value for \(g\) was not the same as the accepted value. This may have been due to human error.”

With all due respect to Alexander Pope, this sort of statement is not very useful. Whenever we compare the results of two measurements of the same quantity, the discrepancy between the two measured values is not very informative without some sense of the uncertainty in each value. No measurement is free of uncertainty, and a discrepancy should only be surprising when it is appreciably larger than the uncertainty in either measurement.²Nor are theoretical predictions free of error. To predict the gravitational attraction between two bodies, for instance, we need to know the value of the gravitational constant, which is itself a measured quantity subject to error. The mathematical framework that scientists use to quantify uncertainty is called error analysis, and the cover of the standard textbook by John Taylor underscores the dire consequences of allowing errors to propagate willy-nilly.

You may object that you don’t have to worry about any of this when two measurements agree. It’s true that in everyday life one can usually get away with ignoring error analysis. But we need to be careful about what we mean by “when two measurements agree.” Take a recent high-profile result in particle physics: experimentalists at Fermilab announced in April 2021 that they’d measured the so-called \(g\)-factor of the muon to be 2.00233184122, while the theoretical prediction was 2.00233183620. Those numbers look pretty damn similar, but both the measurement and the prediction were sufficiently precise that this small discrepancy was quite significant, and the result made headlines around the world.³Most of these articles unfortunately weren’t very good. Natalie Wolchover’s article for Quanta and Matt Strassler’s blog post are notable exceptions.

At the other end of the spectrum there’s the example I started with. Surely if your answer is 3 and the true value is 4, that’s just wrong, no? Again, it depends on the question. Perhaps the ghost of Enrico Fermi appears before you and demands that you estimate how many Americans live in Oklahoma. The true value is 4 million, but 3 million would be a good guess, and will surely placate the restless spirit.⁴I would have guessed 3 million myself. Oklahoma City and Tulsa are both major metro areas (OKC has an NBA team), and southern/western cities tend to be big and sprawling, so I’ll guess they each have close to a million people. I used to live in Colorado, whose population is somewhere over 5 million, and my gut tells me the population of Oklahoma is significantly smaller. So I expected the answer would be somewhere between 2 million and 4 million. If forced by a revenant to pick a single number, I’d choose the middle of this range.

But I’m getting off topic. Suppose you overhear somebody answer “3” to the question “what is 2 + 2?” How does that make you feel?

The poet and classicist Anne Carson considers this question in a poem called Essay on What I Think About Most. Error, it turns out, is what Anne Carson thinks about most, and she explores the subject using a fragment of ancient Greek lyric poetry that I’ve reproduced in its entirety below:

[?] made three seasons, summer
and winter and autumn third
and fourth spring when
there is blooming but to eat enough
is not.

The poet, as Carson puts it, does not seem to know that 2 + 2 = 4. On top of this arithmetic error, the fragment violates the rules of ancient Greek grammar and breaks the convention of its own poetic meter. It’s tempting to chalk all this up to imperfect transmission of the original text over the past twenty-seven centuries. But Carson maintains that the poem is more interesting if we don’t explain away these errors: they remind us that “the fact of the matter for humans is imperfection.” Orderly systems we impose on the world inevitably break down, and our models of reality are always incomplete; we should not confuse the map for the territory. Indeed, it’s only through discrepancies, which always feel like mistakes at first, that science can progress at all.

Let me pause for a second. What’s going on here? Notwithstanding its April 1^st publication date, this post is entirely earnest. A few months ago, my friend Hillel Wayne came up with a plan to “reclaim April Fools’ Day,” as he put it. I’ve enjoyed my fair share of April Fools’ jokes in the past, but I really liked his pitch for a designated day for bloggers to step outside their self-imposed boundaries and write things they otherwise wouldn’t. You can read Hillel’s manifesto and find links to posts by other people who participated in this project here.

I hope I’m not betraying the spirit of Hillel’s proposal by smuggling my atypical content into a post that begins with a staple of the undergraduate physics lab curriculum, but there’s a method to my madness here. In the rest of this post I’ll indulge in a more subjective sort of error analysis, which will take me from art through humor and eventually back to physics.

Anne Carson proposes that the practice of poetry itself is “the willful creation of error.” But it’s not just poetry. Many of my favorite works of art employ error in one form or another to great effect.

Sometimes, as in Carson’s poem, error itself is a major theme. Tom Stoppard’s play Arcadia interweaves two narratives that unfold in the same location 200 years apart, and explores both the errors we make in interpreting the past and the role that error and contingency play in shaping the future. In Nikolai Gogol’s short story The Nose, a nose vanishes from the face of a civil servant, is discovered buried in a loaf of bread, and later becomes a high-ranking official who denies any association with the now noseless protagonist. Something has clearly gone wrong, but characters and readers alike are at a loss to explain exactly what, let alone how.

In other cases, error plays a crucial role in how we experience a work of art. The song Yesterday, by the hip-hop duo Atmosphere, derives its emotional impact from the listener’s initial misapprehension of its subject. The best Coen brothers filmsare⁵Not an intentional error, but in the spirit of the post I’ll let it stand. funny until, suddenly, they’re not, and the jarring moments in which we’re forced to recalibrate our expectations are among the most memorable parts of these movies. In these examples error is an ephemeral phenomenon: we are momentarily disoriented when we realize we’ve misinterpreted or misclassified a work of art, but then quickly latch onto a new perspective that makes our experience legible again.

There’s something exhilarating about this moment of transition that in the right context can elicit sheer delight. Nowhere is this more evident than in the domain of humor. With apologies to Hillel, the traditional April Fools’ joke is a prime example.⁶As you may have guessed, my aim in structuring this post was to evoke a similar response, albeit without “just kidding” as the resolution. Another good example is this pretty terrible joke that made me laugh out loud when I first saw it on a T-shirt in college:

We’re primed to interpret the spatial arrangement of the figures as purely symbolic, but the image doesn’t make sense until we realize that it’s to be understood as a spatial relationship after all. The transition here is immediate and involuntary, but it’s essentially the same process.

This visual gag works exactly once, but for whatever reason, analogous linguistic constructions remain delightful to me every time I think of them. Garden path sentences exploit syntactic ambiguity to lead us briefly into confusion before we find our bearings. My favorite example is “British left waffles on Falkland Islands,” which apparently was an actual headline in The Guardian in the 1980s.

Not all error, of course, is ephemeral. In life and in art, we’re often faced with error that resists resolution, but this ambiguous state is not always distressing. As far as I’m concerned, nothing exemplifies the pleasures of persistent error better than my favorite limerick:

The once was a girl named Marie
whose limericks stopped at line three
when asked why this was

This violation of the entirely arbitrary conventions of limerick form is so viscerally frustrating that I experience it as a sort of bodily sensation, and the delight it brings is inseparable from this frustration.⁷I believe I came up with this one on my own, inspired by “There once was a man from Peru // whose limericks stopped at line two.” Other variations that presuppose knowledge of the two-line limerick include “There once was a man from Verdun” and “Have you heard the limerick about the emperor Nero?” But I must say I like my standalone three-line version best.

This blog is supposed to be about the process of doing physics. I’ve already talked about the quantitative analysis of measurement error, but at the end of the day physics is a messy human activity in which things can go wrong in countless ways. I can speak specifically to the kind of experimental physics that occupied most of my time for the past decade. I struggled against laser beam misalignment, fried amplifiers, faulty cables, bugs in data acquisition and analysis code, electromagnetic interference of all sorts, helium leaks from microscopic cracks in a cryostat, flooding, power outages — I could keep going, but you get the picture. At times it felt like a miracle that anything worked at all.

My research involved a lot of twiddling knobs and interpreting signals on electronic displays. Occasionally — to be honest, quite often — I’d see something utterly baffling, and when I couldn’t figure out what was going on, my muscles would tense up and I’d pace back and forth, muttering things like “What the fuck! How is this not fucking working? This is not physically possible!” I’d spend some time systematically eliminating possible problems, give up in despair, then run through mental maps of the mess of cables connecting different pieces of equipment, repeatedly thinking I might have figured out the problem only to see it persist.

And then, in one glorious moment, the feeling of error giving way to solid ground, accompanied by the same sort of delight I get from that fleeting moment in other contexts. Of course it wasn’t working before; in retrospect, it should have distressed me if it had worked!

This isn’t a Eureka moment in the classic Archimedean sense. I didn’t discover anything about nature, just stopped my experiment from not working in one of the many ways that it was not working at that time. It’s only when the experiment isn’t not working in any known way that the actual physics — the stuff you publish in journals — gets done. The process of getting there, full of pitfalls, dead ends, and errors of all shapes and sizes, can be a source of immense stress — I don’t want to downplay that at all. But among all those errors are moments of intense delight that are absent on the rare occasions that everything works the first time. At least for me, this was a big part of what made experimental physics so satisfying.

I’ll leave you with another excerpt from Anne Carson:

For a poor Spartan poet with nothing
left in his cupboard
along comes spring
like an afterthought of the natural economy,
fourth in a series of three,
unbalancing his arithmetic
and enjambing his verse.
Alkman’s poem breaks off midway through an iambic metron
with no explanation
of where spring came from
or why numbers don’t help us
control reality better.

Scientists and philosophers sometimes ponder, with good reason, what has been termed “the unreasonable effectiveness of mathematics.”⁹You may want to check out these webcomics for a dissenting opinion. Yet numbers only help us understand and ultimately control reality when we confront the cases where they fail us. Don’t underestimate the power of a good error.