Now that we've brainstormed our sources of bias, how do we measure them? Let's make the results of our experiments as accurate as possible!
Welcome back to our discussion of experimental error with a physics tutor. Whenever you are conducting experiments in your high school physics or college physics courses, you need to consider the sources of error that might throw off your result, and you should at least be aware of the methods you might use to mitigate the effects of these errors. An appreciation of fundamental experimental techniques and terminology will also serve you well as you hone your study skills or prepare to tackle the physics GRE or SAT2.
Let’s pick up where we left off! In our last post, we began discussing the sneaky problem of bias in your experiment. Unlike noise, which causes your measured value to jump randomly from run to run, bias, which skews all of your results in the same direction, is more difficult to see in the data. We used the example of trying to measure the height of the average male basketball player. If you measure every player in the country but forgot to ask them to take off their shoes before stepping up to the measuring tape, then all of your measurements, as well as your final result, will be high by an inch or more.
So then, how do we make sure that we don’t forget the shoes or catch our mistake after the fact? If some gremlin in your experiment is throwing off all of your data by the same amount, then how do you detect this shift?
The first step to fighting bias is identifying it. Even though this may seem funny coming from a physics tutor, for this first brainstorming step, let your imagination take over! Try to look at everything in your experiment with fresh eyes. In our last post, we returned to our original experiment – measuring g by timing how long a ball takes to fall from a fixed height – and brainstormed a number of ways in which our measurement of the time could be biased by how we are performing the experiment.
Now that we have drawn up our list of possible culprits, how can we go about measuring how much of an effect each source of bias has on our result? Let’s look at two methods you can use.
Method 1: Switch it up!
When you are trying to test whether some part of the experiment impacts your result, often the easiest thing to do is to try different variations of the experiment that change up the part in question. For instance, there’s the question of whether you or your partner has faster reflexes and if that could bias your results. There’s a simple way to answer this question: switch places! Are you saying “go” and your partner is reacting? Try having your partner say go! Who drops the ball and who clicks the stopwatch? Try it both ways! Notice also that many of these choices are independent; that is, there are four different combinations of you and your partner holding the ball or the stopwatch, and saying “go” or reacting, and you can explore each one.
If you try enough permutations and are careful to keep track of what parameter you are switching, then you will begin to get a feel for which effects matter and which don’t. It might very well turn out that it doesn’t matter if you drop the ball and say “go” or your partner drops the ball and say “go,” but it matters a great deal (maybe 0.2 s or so) if the person holding the stopwatch or the person holding the ball says “go.”
This seems like an awful lot of work, though, doesn’t it? Well, the truth is that it is. Take for example the decades-long quest to measure the electron dipole moment, which is essentially asking if the electron’s mass is centered at precisely the same point as the electron’s charge. These experiments, including the current record holder and a collaboration that is gunning for the lead are absolutely state-of-the-art precision measurements. They might spend days or weeks running the experiment at full speed in order to collect enough data to drive down their statistical uncertainty (effect of noise), but they spend years trying to nail down their systematic uncertainty (effect of bias).
In the record-holding experiment, for example, they identify 9 different quantities, things like their electric and magnetic fields, that could conceivably influence their result. They come up with 512 different permutations of these parameters and test every single one. This is essentially a sophisticated (and thankfully automated) version of the procedure I propose for our humble measurement of gravity.
Method 2: Worst-case scenario
What if there is some part of the experiment that you just can’t switch off? Some approximations – that none of your strings have mass, none of your wires have resistance, and there is not a molecule of air in your physics classroom – can never (within reason for an introductory physics class) be completely true.
Then how do you deal with this inevitable imperfection, the effects of which are probably (but might not be!) negligibly small? What do you do if you want to measure something small with precision? You make it bigger!
Take the question of air resistance. What if the surface of your ball is so rough that the air resistance noticeably slows the ball as it falls? This would result in a longer drop time and an incorrectly low estimate for g. How can we make this effect much worse? We could try placing a fan below the ball blowing up – that air resistance is surely much greater than what we would see with still air. That gives us one bound for the effect of air resistance: the time we would measure if we were dropping the ball in a vacuum should be no greater than the time we measure when fighting against a headwind.
What about the other bound? Let’s try moving that fan so that it blows down on the ball: the time we would measure in a vacuum should be no shorter than the time we measure when being pushed along by a tailwind. If the times we measure under these different circumstances are the same, down to the uncertainty imposed by drop-to-drop noise, then there we go: air resistance probably does not matter. If they differ by, say, 0.2 s (after accounting for noise), then we can say with confidence that air resistance in still air doesn’t push our result one way or the other by more than this 0.2 s.
You can find a way to make almost anything about your experiment worse. Worried about the mass of that string? Try using a thicker one. Worried about resistance in that wire? Try using one five times as long. We spend a good deal of time trying to make our experimental setup better, so go ahead and have fun making it worse.
Putting it all together
By now, you have a good idea of how much thought can go into making even a simple measurement as accurately and precisely as possible. Broadly speaking, we can perform the experiment many times and average the results to drive down the impact of noise on our result as much as possible. We can be creative with how we perform the experiment in order to tease out the various sources of bias that might be lurking in our experimental procedure.
This process gives us a result and uncertainty for one quantity: the time it takes for the ball to fall to the floor, in this case. In the next post, we’ll look at how you take the results and uncertainties for every quantity you measure during the experiment and combine them to get your final result. Stay tuned!