I was reading this article (Edwards & Roy, 2016) on maintaining scientific integrity in the 21st century, that was published a couple of days ago at the journal of Environmental Engineering Science. The facts discussed in the article, for example that scientific output (as measured in numbers of articles per year) is increasing , aren’t necessarily new, the focus of the authors is in discussing what is the impact of such trends on the overall scientific climate. The worry is that the ‘publish or perish’ culture leads to ‘perverse incentives’ that encourage the cutting of corners to stray into the territory of research misconduct.
The authors of the article cite several studies that have found manipulation of impact factors by journals, p-hacking by researchers, and rigged peer-review, as examples of this on-going and worrying trend in science. They reason that perverse incentives leads to reduced scientific productivity, since flooding the literature with fraudulent and misleading results would cause a high experimental error rate as results fail to replicate. The right path to strike between the two extremes of quantity and quality, the authors argue, is one that encourages the best of both worlds.
None of this is news to those of us who have been following the social world of science.
However, what really struck me was this part of the article:
While there is virtually no research exploring the impact of perverse incentives on scientific productivity, most in academia would acknowledge a collective shift in our behavior over the years (Table 1), emphasizing quantity at the expense of quality. This issue may be especially troubling for attracting and retaining altruistically minded students, particularly women and underrepresented minorities (WURM), in STEM research careers. Because modern scientific careers are perceived as focusing on “the individual scientist and individual achievement” rather than altruistic goals (Thoman et al., 2014), and WURM students tend to be attracted toward STEM fields for altruistic motives, including serving society and one’s community (Diekman et al., 2010, Thoman et al., 2014), many leave STEM to seek careers and work that is more in keeping with their values (e.g., Diekman et al., 2010; Gibbs and Griffin, 2013; Campbell, et al., 2014).
Thus, another danger of overemphasizing output versus outcomes and quantity versus quality is creating a system that is a “perversion of natural selection,” which selectively weeds out ethical and altruistic actors, while selecting for academics who are more comfortable and responsive to perverse incentives from the point of entry.
It is also telling that a new genre of articles termed “quit lit” by the Chronicle of Higher Education has emerged (Chronicle Vitae, 2013–2014), in which successful, altruistic, and public-minded professors give perfectly rational reasons for leaving a profession they once loved—such individuals are easily replaced with new hires who are more comfortable with the current climate. Reasons for leaving range from a saturated job market, lack of autonomy, concerns associated with the very structure of academe (CHE, 2013), and “a perverse incentive structure that maintains the status quo, rewards mediocrity, and discourages potential high-impact interdisciplinary work” (Dunn, 2013).
(Please see the full article, which is open-access, for the full references included above.)
What stood out to me about that paragraph was just how well it resonated with my own experiences of academia.
At this point, I have been in academia for seven years: spending four years as an undergraduate student, one year as a Master’s student, and now two years as a PhD student. During this period, I have undergone a transition that has surprised even myself, because it is that drastic.
I grew up wanting to be a scientist. I loved everything about biology, being that girl who’d gross everyone out by having no fear of frogs or other crawling critters, and who’d need to test any trust claim that came her way, taking no claim at face value without proper investigation. In secondary school, I remember that we had a temporary teacher, who, during a lesson in chemistry, asked the classroom who had the career aim of winning the Nobel Prize. I was the only one to raise my arm, and I remember with what pride I did that. I wanted to be a scientist and to discover things that would change the world.
Starting my undergraduate degree, I soaked everything up like an intellectual sponge. I went over and beyond what was expected of me and I remember that one tutor noted (incredulously) on an essay I’d submitted that “40 references for a 1’500 word essay” was a “bit much”. I begged to disagree, since I had done a lot of reading of the primary literature to synthesise the essay, and I wanted to give all authors their due. Similarly, my emphasis on scientific rigour led a senior scientist to proclaim this as “bordering on the fanatic”.
Nearing the end of my undergraduate degree, having completed five lab-based internships and one final-year research project, something had changed. I’d started to get frustrated with science, having developed an intense dislike for the culture where, just like everyone says, quality had been superseded by quantity. This is not because I had any bad practical experiences, far from it – all the labs I ever interned in were full of lovely people – but because of a strong sense of disappointment that science wasn’t as idealistic as I’d imagined and expected it to be.
I’ve always held scientists in an almost super-human regard. I expected them all to be maddeningly intelligent and resourceful, and honest, most of all. To me, being a scientist meant something over and beyond everything else; to me, being a scientist meant that you had dedicated your life to the pursuit of truth. It was bit a reality-check for me to realise that scientists are just people – people who don’t worry too much about the cutting of corners and mis-citing the literature and discouraging the publication of negative results. Over time, these realities chipped away at my reverence, and now, another three years down the academic line, I feel unbearably disappointed by the whole enterprise.
As a friend very eloquently put it to me at one point, I am an “idealist about motives”. For scientists, who claim to be after the truth, the cutting of corners proved to be a reality-check that I simply could not bear.
So that paragraph above resonated with me. I am female (so I’m the ‘W’ in ‘WURM’) and I went into science (the ‘S’ in ‘STEM’) with altruistic motives (to serve society). But the reality of the scientific enterprise and its culture has disappointed me to the point where I simply cannot bear it any longer.
I’m tired of reading papers that cite an interesting finding, only for me to look the reference up and find that it says no such thing. I’m tired of reading papers where they say they used line ‘X’ only to find that line ‘X’ is not in the materials and methods, and the entire paper therefore being a big methodological dead end. I’m tired of blase principal investigators and lab heads who worry more about p < 0.05 rather than the overall likelihood that the finding is genuine, or encouraging experiments to be re-done to allow you to obtain the ‘correct’ result for the paper. I’m tired of the kind of model-driven research, where the model is developed before the results are obtained, and the discussions that invariably follow about how the results can be ‘improved’. And I’m tired of scientists willingly giving up the intellectual rights to their work for the ‘privilege’ of publishing in a respected journal. I’m tired of people going ‘ga-ga’ over the prospect of publishing with Science or Nature, rather than the prospect of being rewarded with the feeling of accomplishment that comes with producing quality work.
More broadly, I’m tired of ‘streamlining’ of scientific education in the interest of saving money. I’m tired of universities proud of being rated ‘top-10’ taking active steps to reduce the quality of the education that they offer. I’m tired of institutions that paint themselves as ‘a great place to work’ that keep you without all the essentials of doing that work well.
As Edwards & Roy (2016) say:
The [‘academic excellence’] rankings rely on subjective proprietary formula and algorithms, the original validity of which has since been undermined by Goodhart’s law—universities have attempted to game the system by redistributing resources or investing in areas that the ranking metrics emphasize.
I’m tired of all that – and so many other things. There’s something very rotten at the heart of academia, and of science itself. I’m tired of the fact that people get away with doing all of these things.
I’m currently in the third year of my PhD, and I’m so looking forward to finishing, so I can leave academia behind, to, as the article says “leave STEM to seek careers and work that is more in keeping with [my] values.”*
*Although this has put me in the awkward position of trying to change career path with a CV that says nothing but ‘biology-biology-biology’. I might write another blog post on this.
That’s why the article on maintaining scientific integrity in the 21st century resonated with me.
- Edwards Marc A. & Roy Siddhartha. Academic research in the 21st century: Maintaining scientific integrity in a climate of perverse incentives and hypercompetition. Environmental Engineering Science. September 2016, ahead of print. doi:10.1089/ees.2016.0223.