Physics Tutor: Doing the Experiment - Discovering Your Bias

physics

Cambridge Coaching

Welcome back to the series of posts on experimental error! As a physics tutor, I am following on my first post, which explored some of the ways even a fairly simple experiment is prone to unavoidable experimental error.  In this second post, we looked at noise and the statistical methods that you can use to mitigate its impact on your final uncertainty.

Noise is random, bias skews your data

Now, though, let’s turn to bias, noise’s more pernicious cousin.  When you take a measurement, there are generally two ways in which the value you are measuring does not accurately reflect the quantity you are trying to measure.  The first is noise, an offset that varies randomly from run to run.  The second is bias, which skews all of your measurements in one direction away from the true value.

Missing the shoes

To illustrate the difference, let’s return to the example we used at the end of the last post.  Imagine you want to know the typical natural height of a male high school basketball player.  You start off by measuring the heights of your three friends on the basketball team: they are 6’0”, 5’9”, and 6’2”.  You want better statistics, so you go on and measure the rest of the team.  By now, you can say that the average player is around 5’11” and there is a range of heights from roughly 5’8” to 6’4”.  Next, you become really ambitious and measure every player in the country.  Now, you can state with great certainty that the dead-center average player is 5’11.9” tall and there is a 95% chance that some randomly chosen player is between 5’8.3” and 6’5.4” tall.  Beautiful result!

But … did you ask them to take off their shoes?  If not, then each player’s shoes and socks would have added an inch or more of elevation to his natural height and every single measurement you made would have been too big.  This is bias; your dogged pursuit of great statistics took care of the noise, but a mistake in our experimental technique has thrown off our result.

Whenever you perform an experiment, even a very simple one, you should always ask yourself if you are forgetting the shoes.  Is there some effect, some quirk of your experiment, that is throwing off all of your data?

Bias is sneaky

I say that bias is noise’s more pernicious cousin because it is sneaky.  You can look at your data and see the effect of noise.  In the example above, you can immediately tell that you are seeing some natural player-to-player variation after you have measured your three friends.  It might take much more work to quantify your noise (more measurements, taking an average, maybe fitting a distribution), but you can at least see that the noise is there.

Looking at just these three measurements taken under the same conditions, though, doesn’t tell you anything about your bias.  All three heights might be too large by an inch or too small by three inches, but you would have no way of knowing.  Even after sampling the entire nation, you would have no way of detecting your error if you didn’t have some standard answer to compare your result against.  If bias is so hard to detect, then how do we go about fighting it?

Examine your biases

Now let’s return to the experiment we described in the first post on experimental errors: we are trying to measure g, the acceleration due to gravity, by measuring how long it takes a ball to fall from a given height.  Specifically, we focused on the act of using a stopwatch to measure the time that elapses between when your partner lets go of the ball and when it hits the floor.  We performed a quick analysis of the noise that we might expect to see in our data due to our slow and imperfect trigger fingers, but that was the easy part: this source of error was fairly obvious and we could easily average over many runs.

Then we started throwing out questions that could potentially confound our measurement.  What if you say “go” and start timing, then your partner drops the ball some time later?  What if your partner is the one who says “go” and you are the one with the delayed reaction?  Would that bias the time you measure one way or another?  What if you watch the ball as it falls and anticipate when it will hit the ground?  What if, what if, what if?  In even such a simple experiment, we can find a multitude of ways in which our method might push our result one way or another away from the true value.

Whenever you are trying to weed out your biases, the first step is usually to sit down, think critically through your experiment in gory detail, and just brainstorm things that could go wrong.  Once we have a list of possible culprits, how can we go about determining how much they are throwing off our result?

In the next post, we’ll take a look at two common techniques that are used to explore the biases inherent in your experiment and to quantify how much they might be hurting your result.

Comments

topicTopics
academics study skills MCAT medical school admissions SAT expository writing college admissions English MD/PhD admissions strategy writing LSAT GMAT GRE physics chemistry math biology graduate admissions academic advice ACT interview prep law school admissions test anxiety language learning premed MBA admissions career advice personal statements homework help AP exams creative writing MD study schedules test prep computer science Common Application summer activities history mathematics philosophy organic chemistry secondary applications economics supplements research 1L PSAT admissions coaching grammar law psychology statistics & probability legal studies ESL CARS SSAT covid-19 dental admissions logic games reading comprehension engineering USMLE calculus PhD admissions Spanish mentorship parents Latin biochemistry case coaching verbal reasoning DAT English literature STEM excel medical school political science skills AMCAS French Linguistics MBA coursework Tutoring Approaches academic integrity chinese letters of recommendation Anki DO Social Advocacy admissions advice algebra art history artificial intelligence astrophysics business cell biology classics diversity statement gap year genetics geometry kinematics linear algebra mechanical engineering mental health presentations quantitative reasoning study abroad technical interviews time management work and activities 2L DMD IB exams ISEE MD/PhD programs Sentence Correction adjusting to college algorithms amino acids analysis essay athletics business skills careers cold emails data science dental school finance first generation student functions graphing information sessions international students internships logic networking poetry resume revising science social sciences software engineering tech industry trigonometry writer's block 3L AAMC Academic Interest EMT FlexMed Fourier Series Greek Health Professional Shortage Area Italian Lagrange multipliers London MD vs PhD MMI Montessori National Health Service Corps Pythagorean Theorem Python Shakespeare Step 2 TMDSAS Taylor Series Truss Analysis Zoom acids and bases active learning architecture argumentative writing art art and design schools art portfolios bacteriology bibliographies biomedicine brain teaser campus visits cantonese capacitors capital markets central limit theorem centrifugal force chemical engineering chess chromatography class participation climate change clinical experience community service constitutional law consulting cover letters curriculum dementia demonstrated interest dimensional analysis distance learning econometrics electric engineering electricity and magnetism escape velocity evolution executive function freewriting genomics harmonics health policy history of medicine history of science hybrid vehicles hydrophobic effect ideal gas law immunology induction infinite institutional actions integrated reasoning intermolecular forces intern investing investment banking lab reports linear maps mandarin chinese matrices mba medical physics meiosis microeconomics mitosis mnemonics music music theory nervous system neurology neuroscience object-oriented programming office hours operating systems