Publishing in a Q1 Journal is not Q1 Research

Having only been in academic research for a few years as a student it has been strange to find such applause for publishing in “high quality” Q1 journals. Metrics are easy and quantifiable ways to act as proxy measures for quality or impact, but going beyond the first layer it does not hold up. Individual research outcomes should be priority when assessing quality.

Quartile rankings place journals into different equally distributed categories:

• Q1 = Top 0-25% of journals
• Q2 = 25-50% of journals
• Q3 = 50-75% of journals
• Q4 = Bottom 75-100% of journals

As each quartile category contains an equal number of journals, it would be natural to assume that publishing in a Q1 journal would indicate that research is among the top 25%.

This seemingly is far from the truth.

I took the publicly available journal rankings for 2024 from Simago to examine the distribution of publications across these different ranked quartiles. There were around 30,000 journals in 2024 with more than 9 million publications in the preceding three years. A normal expectation would be that each quartile of journals would publish 25% of the research or that 2.2 million articles would come from roughly 7,500 journals.

The top Q1 journals published more than half of all publications.

• Q1 ranked journals had 53% of publications
• Q2 ranked journals had 25% of publications
• Q3 ranked journals had 14% of publications
• Q4 ranked journals had 9% of publications

This is Goodhart’s Law in action. “When a measure becomes a target, it ceases to be a good measure”. As Q1 journal rankings have become a target for researchers, it has diluted the ability to assess the quality of research in these journals. Ranked by whatever metric, 53% cannot fit into the top 25%.

This holds true for subgroups.

Filtering journals to those with in the area of medicine we find that once again, more publications than to be expected. The top Q1 journals over represents publications:

• Q1 ranked journals had 42% of publications
• Q2 ranked journals had 23% of publications
• Q3 ranked journals had 17% of publications
• Q4 ranked journals had 18% of publications.

Quality cannot be assessed by a single metric. In the last three years if you were to select a publication at random, you’re most likely to pick an article published in a Q1 journal. Similarly, if you were to select a publication from a Q1 journal at random, it is more likely to not be Q1 research than is.

Journal quality is not a measure of research quality.