- Practising Development aims to explore ideas, discuss issues and share learning around research, information and development. Managed by INASP, the views and opinions expressed on Practising Development are those of the individual authors and do not represent those of the organisation.
- Subscribe via RSS
Tag Archives: Scientific method
Since returning from a workshop on science communication for journalists I have been thinking a lot about science journalism both in the UK and abroad and wondering if science journalists need to understand the scientific process in order to write good science stories. A number of people will argue that no, you don’t need to understand science to write a good science story — after all you don’t have to be an economist to write on economics or an investment banker to write on finance. While I tend to sympathise with this position I actually disagree with it and here I will outline why.
Reason 7: Part of the dataset was suppressed In the examples I mentioned yesterday (Reason 6: The entire dataset was not considered) I talked about research which does not get published simply because the incentive to publish it is not high enough. This happens frequently but unfortunately, sometimes more sinister factors are at play. Some pharmaceutical companies have been accused of deliberately suppressing findings which suggest that their drugs do not work. This phenomena is well known and there has been a lot of action in recent years to ensure that the results of clinical trials are more accessible. In some countries, anyone conducting a clinical trial is compelled to register it on a clinical trial registry. This means that anyone wanting to review the evidence will be able to find the results — including those which gave results which contradict drug company claims. Next — Reason 8: Their belief was … Continue reading
Reason 6: The entire dataset was not considered The scientific method is a good way to get more objective answers to questions. However, the way that we publish scientific findings is rather less objective. Scientific journals are not all equal. Some are seen as more ‘sexy’ than others. The measure of a journal’s ‘sexiness’ is its impact factor; so journals with high impact factors are seen as the best to publish in. Scientists are judged (by promotion committees, funding agencies etc.) according to the impact factors of the journals they have published in. Unfortunately, this sets up a bias against publishing negative results. Positive results (where you prove that something works) are intrinsically more ‘sexy’ than negative results. Therefore scientists are much more likely to get results published (particularly in high impact journals) if they have found a positive result. There is a much lower incentive to publish your negative … Continue reading
Reason 5: Someone else misinterpreted the results Yesterday I mentioned that scientists sometimes misinterpret what they observe but of course it is not only scientists who interpret scientific findings. Journalists, who wish to make scientific findings into compelling news stories are sometimes guilty of misinterpreting (and sometimes downright misreporting) scientific findings. So for example, a study which demonstrates that individuals who consume a small amount of chocolate every day have a lower risk of developing a certain type of cancer, may be reported as ‘Scientists prove that chocolate is good for you!’. Of course this headline does not accurately convey what the study proved. Unfortunately, in a few months’ time, if another study shows that eating chocolate is positively correlated with a different disease, you might get another headline announcing ‘Scientists say chocolate is bad for you!’. In this case the scientific findings of both studies may well have been … Continue reading
Reason 4: The scientist misinterpreted the results Sometimes the scientific findings are correct but the way they are interpreted is not. I can give an example from my own scientific career as an immunologist. I used to study the response of a certain type of human blood cell (an NK cell) to malaria parasites. In one study, using microscopy, we were able to show that some NK cells attached to some malaria parasites. In the paper we wrote to report this, we speculated that perhaps the NK cells were in fact directly communicating with, and becoming activated by, the malaria parasite via a structure known as an immunological synapse. The reason we speculated that this might be the case was that if that was true it would have been extremely exciting (at least to nerdy immunologists like us!). However, subsequent experiments suggested that while our observation was correct (that some NK … Continue reading
Reason 3: They lied (or at least stretched the truth!) Let’s be completely honest. Scientists are human beings and human beings lie. There are numerous cases where the results that scientists report are in fact fabricated. Some of these cases are high profile — such as the fraud committed by Korean scientist Hwang Woo-Suk. In addition to blatant fraud, there are also plenty of cases where scientists unconsciously inflate effects because they believe that they exist. There is a detailed (and sometimes quite amusing) record of scientific results which have been retracted for various reasons online.