Should My PhD Exist?

My PhD is important for the wrong reasons.

Titled “On examining poor statistical practices in published clinical prediction models” for the last 273 days I have been searching for problems and issues in prediction research.

What I rarely see are, well developed, rigorous prediction modelling studies that are pre-registered with appropriate methods, following expert guidelines and published with open access code/data for reproducibility and replicability. Simple, good research practices.

That is not to say there aren’t plenty of fantastic, impactful studies that change practice and policy. But after nearly a year, I have found there is more than enough sub-optimal research, that I am spoilt for choice, having had to sideline countless study ideas. And it makes me wonder…

Should anyone be spending time and resources uncovering and highlighting problems in health and medical research?

Yes. Because these issues exist, are relevant to patients and need to be mitigated.

But should problematic studies have been conducted with more rigor in the first instance?

Yes.

So, how do issues creep into published health and medical research?

I see there are two possibilities. Studies are unknowingly or knowingly published with problems. Bad actors who knowingly publish false, misleading or poorly executed studies are likely to continue, while a form of ‘profit’ exists – being financial, career gain or prestige for example – and this is unlikely to change.

Studies which are published with problems undenounced to the authors however, could arise from a lack of domain knowledge – an inappropriate statistical method for the data or research question – or something as trivial as human error. The bias for statistical significance is a common hacking practice and an example where articles have likely been published with an unknown problem – that the authors may be unaware to the negative implications of statistical hacking or that they are engaging in it.

One of the best ways to produce high quality research with integrity is by pre-registering well developed studies, mitigating bias, hacking practices and producing transparent research. Clinical trials are required to be registered and is considered a “scientific, ethical and moral responsibility” by the World Health Organisation. While it appears that quantity has become the aim of the game, quality research should always be the focus.

So, should my PhD topic exist?

Ideally no, but we do not live in an ideal world and problems exist that need to be addressed. My hope is one day science will be so rigorous, that all focus can be on progressing science forward, not identifying common problems. But until issues are worked out, there is a need to identify and tackle them.