Peter’s Axioms on Quoting Statistics

Go to any article on same-sex marriage and the controversy around gay unions, and at some point someone is going to make a claim and back it up by citing an academic paper. Knowing that many of you do that, here’s some little “Stats Axioms” to help you out.

  1. Make sure the academic paper exists – Seriously folks, make sure it exists before you copy the reference off someone else’s website. It is HUGELY bad form to just copy a quote from someone and then find out 24 hours later that the paper doesn’t even exist, that someone else made up the reference a decade ago and that everyone has been copying that ever since.
  2. NEVER rely on secondary sources – Unless you read the paper itself (and not just one person’s report of it) you shouldn’t cite it, let alone copy someone else’s report of it. Make sure you have actually seen the real thing in your hands or on your screen. It is not a valid excuse to say “I just relied on what xyz said”.
  3. READ the primary source. Read it again. Read it again. Make notes.
  4. Represent the primary source ACCURATELY and fairly. Remember, other people will read the paper (even if you haven’t) and they will pick up any teensy tiny exaggeration you make or the details you pass over that actually contradict the point you are trying to make.
  5. If the paper is part of a series of papers, READ THEM ALL. Read them all again. Longitudinal Cohort studies often find different results as the research progresses, results which might change those initially reported or clarify other findings.
  6. If you are using the research to claim a comparison between group A and group B, MAKE SURE the research actually covers group A and group B. Yes, you might find something bad about Group A, but if another bit of research says that Group B is just as bad on this particular issue, you’re going to look very silly when you claimed that the original research showed how heinous Group A was.
  7. NEVER FORGET that there are people out there who can calculate standard deviations in their head. Do you even know what a standard deviation is? Are you prepared to defend dismal sample sizes when challenged? Did you even read the paper to find out what the sample size was? I have sometimes dumped papers rather than citing from them because once you examined the sample sizes you realised the confidence intervals were so wide as to be meaningless.

There you go. If you follow these simple rules then you’re going to get 4 or 5 out of 5 every time on my handy Stats Watch ranking! If you don’t, you’re likely to get torn apart as we all see that you didn’t actually comprehend what the research actually told us and (and this is the absolutely worse crime in using statistics) you simply manipulated the figures to fit your bias and desired outcomes. Let the reader understand.

Any others anybody would like to contribute?

Tagged with: , , , , , , , , , , , , , , , , ,