There are a couple recent features/editorials in the Nature special section for irreproducible research that discuss our cognitive bias during experiments and whether we take any steps to limit those biases.

A recent Nature News article (Poorly designed animal experiments in the spotlight) highlighted that animal studies suffer from poor design, which is probably also the case for most non-ecology plant studies. Do we ever consider the alternative hypothesis, or focus on results that validate the working hypothesis or model? Do we only question results when they deviate from our expectations? If a trend is not significant but fits our expectations, do we add more samples until we obtain a significance? These would all be examples of our bias.

There are solutions, but how many of us randomize experiments? Blind the results before analysis? Calculate the sample size required for desired power?

These articles are interesting and thought-provoking, especially in the current climate of research often found to be irreproducible.

Let’s think about cognitive bias

How scientists fool themselves – and how they can stop

Blind analysis: Hide results to seek the truth