Review: The Immortal Cell

I bought a second-hand copy of The Immortal Cell: Why cancer research fails by Gerald B. Dermer on a whim. The book is out of print, but I had seen someone on a bulletin board recommend it because of its criticism of the use of cell lines in cancer research. I love reading critical (but sensible) accounts of ideas/techniques that have gained mainstream acceptance, so I thought the book would be worth a read.

Three caveats:

1. The book was written in 1994 (24 years ago!).
2. It is obvious from reading the book that the author has an axe to grind — being motivated by equal parts frustration and resentment towards cancer research.
3. Although I have a firm biological background, I am not an expert on cancer research. I think it’s important when approaching a book like this to be aware of your previous knowledge, to allow you to assess the plausibility of the arguments, while also acknowledging that some arguments might be relevant (and some may not).

With these caveats in mind, I approached the book as a historical curiosity. Yet, I found it quite an intriguing read.

The book is a criticism of contemporary research culture. Perhaps more so than it is a criticism of the state of cancer research in 1994.

Reading the book, I was surprised how little our overall understanding of cancer biology has changed in the quarter century since the book was written. Perhaps I should not have been, considering how the central thesis of the book is that scientists asking the wrong questions end up on wild goose chases with little tangible benefit.

I am quite certain that some of the statements Dermer makes have since been proven wrong. In addition, one of the horses Dermer is flogging is the failure for age-adjusted cancer mortality to improve. Although there are issues with over-diagnosis which might skew the results, and the improvements (seem to?) have been modest, I think cancer patients are better off these 25 years down the line since the book was written. This shows that although cancer research is not perfect, it does improve treatment, which is reassuring.

Still, Dermer has an interesting perspective on cancer research. Before entering academia, he worked as a hospital pathologist, and had access to plentiful human tumour tissue samples. Through his work, he realised that his understanding of cancer biology was different from the understanding of cancer biology communicated through the scientific literature. Eventually he realised that this was because he was working with genuine samples of human cancer, while many of his non-pathologist colleagues were working on cell lines — which are quite different beasts.

In essence, the book is a criticism of the use of these cell lines, since they make only poor models for human cancers as they exist in the body. The cells of a living tumour are very different from the pampered, immortalised cells at the bottom of a petri dish. Perhaps, Dermer asks, cancer research using this very different model, gives them a picture of cancer that is very different from the real thing?

I think the book is worth reading if only for this reason: what is cancer cell lines actually do not make a good model for research on human cancer?

As Dermer writes:

If their favourite model [the cell lines] was wrong, then all the information and ideas that have flowed from it were also wrong. The work of tens of thousands of scientists, over a period of more than thirty years, would be invalidated. The cell line model of cancer research is like the foundation card in a house of cards—pull it out and the entire “house” of cancer research will come tumbling down.

Throughout my career, I have always stayed away from cell lines, without really knowing why. Partly, I felt they were too ‘hyped up’ to be attractive to my contrarian personality. Secondly, it makes intuitive sense that cell lines might not be the best models for a beast that’s really quite different. I felt the same about using the fruit fly as a cancer model for my D. Phil. work, and I made sure to ask questions that helped me understand fruit fly biology better — while being very cautious to extend the findings to human biology beyond what was necessary to put my results into context (and even then it felt like bit of a stretch).

To some degree, everything we work with is an approximation of the truth. Words can only convey part of our intended meaning, and models will never be a substitute for the real thing. However, as Dermer acknowledges in the book, cancer cell lines are very convenient to use for molecular biology — which explains why they have become so popular.  Whenever models — approximations — are used, it is however crucially important to never forget that this is all they are: models. What Dermer is criticising is the blind acceptance of cell lines as genuine substitutes of cancerous tissue:

when today’s researchers discuss “lung cancer”, they are usually not referring to lung cancer in the body but to cells derived from lung cancer that have lived for years in plastic dishes in the laboratory. The distinction between cell lines and human cells in vivo is no longer being made, largely because cell lines are so ideal for the methods of molecular biology.

This distinction is still not being made. Therefore, the book’s central argument is still valid. Are we using the right tools for understanding the biology of cancer? Cell lines are but one tool, to be complemented with animal models, primary cell lines, and tumour tissue. Of course, many research papers unite these models, to build a better understanding of cancer biology. However, many of the findings being tested and applied — and the ideas being generated — stem from work on immortalised cell lines and animal models. If the cell lines are very different from genuine human cancers, perhaps the results of such experiments are not very applicable? The same argument can be made for animal models: how good are they — and, but extension, their results?

Cancer has been cured in mice and petri dishes countless times. These are genuine achievements of the scientific process. However, such breakthroughs will be hard to bring into the clinic unless the cures are more generally applicable. The failure of many of these ‘cures’ in the clinic suggests that the type of cancer we model with cell lines and in mice is very different from the types of cancer that are relevant to human beings. This is typical of answers that have been arrived at through asking the wrong questions. (And very similar to current sentiment towards research on neurodegeneration.)

What are the right questions that we should be asking?

For starters, these would concern the models we use: Are we using the best models? The best models are the ones that are the least different from what we are trying to model. If you lost your keys, it is better to look for them where you lost them, rather than under the street light across the road — even if the light is better on that side. If we can’t experiment of living human cancers, perhaps extracted tumour tissue really is the second-best thing? They are not as easy to access or to work with as cell lines, but they are closer to what we are looking for: the ‘keys’. The cell lines in this analogy would be the equivalent of looking under the street light; it’s a lot more convenient, but also pointless. Whatever we find will not be our keys.

Second, out understanding of cancer should be based on genuine human cancers: the type we can study through genuine tumour material. That way, we can understand what cancer is. After that, we can formulate our hypotheses (what is cancer, really?) and test them using the model that would be the most suitable. Cell lines are tools, and I am certain they are suited to answering some questions. Where I agree with Dermer is that they have been given a prominence that is hard to justify: everyone is working with cell lines. Have all of these researchers, independently, asked themselves if these really are the best models to use? Or is an entire research programme looking for their keys across the street, because it never occurred to them to look anywhere else? (It should be noted that Dermer reserves his criticism for molecular biologists, and not pathologists — he seems to believe, perhaps rightly?, that pathologists study the ‘real’ cancer, and therefore have a better understanding of what ‘cancer’ actually is.)

Another interesting thing that stood out to me was how much people knew about cancer already by the 1960’s. Immunotherapy is not a new idea. As a non-cancer biologist, this was news to me. I was also much intrigued by Dermer’s criticism of immunotherapy to treat cancer, since I was of the belief that immunotherapy might success where other treatments have failed previously. Taking some time to ponder the issue, perhaps immunotherapy has been oversold? If the immune system protects against cancer, suppressing it would be expected to promote its development. Rapamycin/sirolimus is an immunosuppressant. Its use in human patients is however associated with a decreased incidence of cancer. Its just one example, but it does suggest that the hypothesis that cancer arises through immune system failure does not explain everything. 

Interestingly, Dermer makes several allusions to the developmental nature of cancer (and rapamycin fits snugly into this mold, by inhibiting TOR: a key regulator of cell development and biology). What he writes about hormones near the end of the book was particularly intriguing, where he discusses the epidemiological findings that early pregnancy in humans (and rats, experimentally), is associated with a decline in the life-time risk of developing breast cancer. This, he says, might be because pregnancy causes the cells in the breast tissue to differentiate — making them less likely to become cancerous. This is followed by a suggestion that a pill that somehow mimics pregnancy (and better so than current hormonal contraceptives) might be protective against breast cancers if taken early in female adulthood, by allowing the developing cells to differentiate. Perhaps someone with better knowledge of the field can correct me if this is wrong (please do!), but this suggestion made sense to me. I will update this post if/when I have figured this one out. 

Stimulating ideas aside, Dermer has an issue with the ‘toxic’ chemicals that compose most chemotherapies. He mentions that some of these are derived from mustard gas, which is pure scare-mongering. There are several parts of the book where he criticises such treatment of cancer, sometimes even suggesting that untreated cancer would make a more dignified cause of death. Such statements show that Dermer is a pathologist and not a clinician, since my understanding (from clinicians) is that untreated cancer is a horrible way to go. It was while reading these sections that I worried that Dermer would best be classified as a maverick crackpot (of which cancer research has several). However, just because not all of someone’s opinions make sense, doesn’t mean that they don’t have anything useful to say. For example, I am reminded of Mikhail Blagosklonny, who is a bit quirky, by all accounts, but who has some provocative ideas about the biology of ageing — including cancer. There are intriguing parallels between the pre-cell line model of cancer (according to Dermer) and the hyperfunction theory of ageing, for example; through the theories’ common emphasis on the role of development and differentiation.

For anyone who is interested in the history of science and/or cancer biology (and with some knowledge of the field(s), and an open mind), I would definitely recommend this book. It is out of print, but a Kindle version is available. It is well-written, and can easily be read in one or two sittings.

Overall, I really enjoyed the book, and it has provided me with some definite food for thought. Perhaps this is confirmation bias, but I think Dermer’s main points are correct: that cell lines make poor models of human cancer, and that it’s important to ask the right questions (by using the right models, or acknowledge their limitations) when doing any kind of research. The latter is common sense. Perhaps we need more of that in science, and in society more generally.

To begin with, we can change the incentives that pervert the scientific enterprise by valuing quantity over quality. Let us make scientific research less of a scramble up the academic career ladder, and more about a job where you get to spend your days thinking about cool ideas and the best way to test them — and then doing just that.

In Defence of the Thesis

For the past two years and some, I’ve been working towards a doctorate degree. As a result, I have been thinking a fair amount about the process of doing a doctorate. I have a couple (several) pet peeves relating to doctoral study and the organisation and administration thereof, but today I would like to share some of my thoughts on the climax of the degree: the thesis, and the writing thereof.

Thesis writing is different between countries. Here, in the UK, the thesis is (typically, traditionally) a stand-alone piece of work (a monograph), that is written from scratch and complete with chapters forming the introduction, listing the materials used, describing the results of the work–and what they all mean in a greater context. In other countries, the thesis can be formed from a collection of scientific papers (each one standing alone with their own introductions and lists of materials, and results and discussions), all bound together with a joint introduction and discussion of the implications of this part of the literature. I think both methods have merit.

Many people in academia prefer the latter method: of collating a student’s professional oeuvre. Students, in particular, like this method because it requires minimum effort on their behalf; all that’s required is for the student to collect all their published papers and write an introduction to their body of work and  to discuss what it all means. Ease aside, another argument in favour of the ‘collated’ thesis is that it won’t take time away from writing research articles; the help the student further their future career. A third argument would be from the examiners, who, typically, prefer shorter theses, since they are easier to examine.

However, personally, I am in the minority group of finding the merits of the monograph thesis to be the most convincing–and I’d like to use this post to explain why.

In a Nature News & Comment article asking What’s the point of the PhD thesis?, several arguments pro and against the prevailing monograph model are presented. The arguments against focus on the need to publish during your PhD, whereas one of the arguments in favour of the monograph is an emphasis on discovery:

But others argue that the pressure to publish could rob PhD students of valuable parts of their studies, such as the time to shape their research path and to think creatively and independently. “The PhD might become driven by papers only,” says Farrar. “Students might end up spending their time focusing only on what papers they can produce, then staple them together with a summary and they’re done — adding to the sense that the whole scientific enterprise is a paper factory rather than an exploration.”

Part of the disagreement of the purpose of the PhD thesis is, I think, because we forget to bring into the discussion the purpose of the doctoral degree itself. Is the purpose of a PhD to publish papers to improve your post-doctoral career prospects? Or is the point of the PhD to pursue an exploratory project and gain research and management skills? As for myself, I believe the purpose of the PhD is the latter: to gain training in long-term project management, and, hopefully, discover something exciting and new.

One of my past professors told me that the different levels of training in academia are suited to different research projects. Undergraduate students are typically in the laboratory for a short time only, and are therefore not expected to produce many results. Therefore, my professor argued, undergraduate projects are a great opportunity to engage in truly exploratory work, where success is rare, but if it does happen, incredibly exciting. Master’s and PhD students are in the laboratory for a longer time, and have the time to get more results. Still, researchers on these levels are still relatively spared from the pressure to publish, and my professor thought the PhD was your last opportunity as a serious researcher to break genuinely new ground and to seize the opportunity to truly explore the unknown. After your PhD, if you decide to stay on in academia, your work has a different purpose: to publish. Therefore, my professor argued that postdocs deserved–needed–safe projects that were guaranteed to succeed. And, indeed, this is the kind of project that most senior researchers prefer, since this is the easiest way to get grants, to allow you to stay on in academia. For the professional academic, there is no future in failure. Unfortunately, failure is often the result of genuine exploration.

Assuming that the PhD is the last exploratory frontier before the reality of academia bites, this has three possible outcomes. One, the PhD student will come across something truly novel–although this is rare. Two, the PhD student will not find anything, because the project was a failure. This is, thankfully, also quite rare. Three, the PhD student will find something, although it is not very exciting. This is, I think, the most common outcome; given the nature of science as proceeding only incrementally.

With these three outcomes in mind, what are the likely implications of these on the final thesis?

I get the impression that the collated PhD is, partly, a result of the way PhD students are seen in research, and a consequence of the contemporary ‘supervisor-student conflict’; where the student and supervisor can have very different intentions for the desired outcome of the doctoral degree. Many laboratories enjoy PhD students as ‘free labour’, since the student’s stipend allows the laboratory to not pay for the student’s upkeep, and stipends also often come with money to fund laboratory consumables. To have a PhD student is therefore a low-cost solution for a laboratory in economic terms. In real terms, the student does however cost the laboratory something: time. A new student takes time to train, and they require supervision and academic mentoring as part of this training. Whereas the student wants this training (indeed, it is the reason why they are doing the PhD in the first place), it is in the supervisor’s interest to minimise this training to increase their own cost-benefit ratio: to benefit from the student’s labour while minimising the time spent on direct supervision.

Genuine supervision will only come if the supervisor is willing to disregard the drain this supervision will place on their time; working altruistically to hope academia sensu lato will benefit from the investment of the supervisor’s time on student training. Sadly, this kind of supervision is not encouraged by the contemporary research climate, and it is likely that PhD students will be pressured to deliver: to deliver results and, ultimately, publications. In this case, the collated thesis will be the outcome, because the focus of their studies was publication. The end result of this kind of training will be to produce students who are good at publishing–but who may not necessarily have received much experience of the more subtle nuances of doctoral work.

Conversely, the lucky students who find themselves in laboratories where discovery is the main goal (and publication a fortunate consequence), and which are run by supervisors who take their training responsibilities seriously, the focus of the PhD student’s training will be on skills development: of learning new methods; of getting a solid understanding of experimental design and data analysis; of getting teamwork experience; of realising just how much fun research can be. This kind of PhD is likely to have limited scientific impact–since the focus is on the process (learning), rather than the result (publication)–although useful, transferable skills will have been learned. This kind of doctoral project is less likely to result in sufficient publications to form a collated thesis, and is more suited for a monograph.

Indeed, it is my belief that the monograph is a training exercise in its own right, and an important part of any doctoral training process. Because in what other path of life (unless you go into a writing career) do you get the opportunity to plan a project on such a grand scale? A monograph thesis in the natural sciences is typically around 50,000 words, and in excess of 200 pages (albeit double-spaced). To write such a document requires a lot of important skills in project management and data interpretation and analysis–in addition to communication skills (regarding writing and data visualisation); analytical skills (regarding the interpretation of the results–or lack thereof); and historical skills (to put the work into context and propose future directions). All of these are skills that should be considered essential to the doctoral training process, meaning that the monograph thesis is a crucial part of any PhD.

In fact, I would go as far as to argue that the monograph thesis is the most important part of the PhD. This is an opinion that has earned me much ridicule among my fellow academics, but I think the doctoral thesis should be considered a masterpiece. Consider the following excerpt from the Wikipedia entry on ‘Masterpiece’:

Originally, the term masterpiece referred to a piece of work produced by an apprentice or journeyman aspiring to become a master craftsman in the old European guild system. His fitness to qualify for guild membership was judged partly by the masterpiece, and if he was successful, the piece was retained by the guild. Great care was therefore taken to produce a fine piece in whatever the craft was, whether confectionery, painting,goldsmithing, knifemaking, or many other trades.

Considering that a PhD is an apprenticeship in science, the thesis–which is the final culmination of the project–would be the masterpiece of the academic training process, and should be representative of the student’s best work; showcasing their skill as a researcher, discoverer, communicator, and qualified academic to the already-academics who have been chosen to judge the merits of the work through the final examination.

Some may argue that the view of the monograph thesis as an academic masterpiece are long gone, and that academia has moved on. And that, again, brings us back to publication. Is the primary purpose of academia really to publish? Or is it to better understand the world? I am enough of an idealist still to believe the latter to still be true–even if academia has forgotten. Therefore I am willing to profess to another unpopular opinion, to which I am alluding in the title of this post, that we need to defend the importance of the thesis. If the thesis is the masterpiece of academic training (which I believe it is), why do we show the thesis so little respect? Typically, the student loathes to write the thesis; the examiners loathe to read and examine it; and the final fate of the thesis is to linger forgotten in a university library depository somewhere–rendering all the work that went into it for nothing. People use this as argument in favour of the collated thesis; arguing that because the thesis has no purpose, let’s make the thesis as simple as possible. But again, this is not the point of the PhD or the thesis. The point of the PhD is training, and the thesis is to show demonstrate what all the training has amounted to. Therefore, I believe we should show the thesis greater deference: let us make the thesis the work on which the doctoral candidate is assessed–not only as part of the examination process, but also in the professional context. Postdoctoral positions should be be awarded primarily on the basis of the candidate’s publication record, but also on the quality of their thesis. because whereas publications are typically authored by multiple people (and the first author may not even have written the publication themselves), the monograph thesis is written by one person only: the doctoral student–making the monograph the best representation of the student’s ability.

I’m leaving academia, because I am too much of an idealist about science to last long in the contemporary academic climate (or to want to). But if I were to stay and eventually make it to the point of running my own laboratory, I would ask postdoctoral candidates applying to work in my laboratory submit a copy of their thesis alongside their more traditional application. Looking them through, I would give preference to the candidates who took care in writing their final thesis, since this is (to me) a sign of a conscientious scientist who actually enjoys what they are doing.

Because at the end of the day, science is about enjoying the struggles of leading up to discovery. And writing is an important part of that, since a good scientist also needs to be a good communicator; being able to explain their work in simple terms to prove that they actually understand what they’re doing. Incidentally, this is another argument in favour of the monograph: don’t go into academia if you don’t like writing. Because writing is what it’s all about–whether you publish or not.

Where is the wind blowing?

I came across this live map of wind patterns across the planet, via the BBC. Really interesting!

And it makes me realise how little I know about meteorological science: why is the wind blowing more over bodies of water than over continents?

 

 

Scientific integrity and why I’m leaving science

I was reading this article (Edwards & Roy, 2016) on maintaining scientific integrity in the 21st century, that was published a couple of days ago at the journal of Environmental Engineering Science. The facts discussed in the article, for example that scientific output (as measured in numbers of articles per year) is increasing , aren’t necessarily new, the focus of the authors is in discussing what is the impact of such trends on the overall scientific climate. The worry is that the ‘publish or perish’ culture leads to ‘perverse incentives’ that encourage the cutting of corners to stray into the territory of research misconduct.

The authors of the article cite several studies that have found manipulation of impact factors by journals, p-hacking by researchers, and rigged peer-review, as examples of this on-going and worrying trend in science. They reason that perverse incentives leads to reduced scientific productivity, since flooding the literature with fraudulent and misleading results would cause a high experimental error rate as results fail to replicate. The right path to strike between the two extremes of quantity and quality, the authors argue, is one that encourages the best of both worlds.

None of this is news to those of us who have been following the social world of science.

However, what really struck me was this part of the article:

While there is virtually no research exploring the impact of perverse incentives on scientific productivity, most in academia would acknowledge a collective shift in our behavior over the years (Table 1), emphasizing quantity at the expense of quality. This issue may be especially troubling for attracting and retaining altruistically minded students, particularly women and underrepresented minorities (WURM), in STEM research careers. Because modern scientific careers are perceived as focusing on “the individual scientist and individual achievement” rather than altruistic goals (Thoman et al., 2014), and WURM students tend to be attracted toward STEM fields for altruistic motives, including serving society and one’s community (Diekman et al., 2010, Thoman et al., 2014), many leave STEM to seek careers and work that is more in keeping with their values (e.g., Diekman et al., 2010; Gibbs and Griffin, 2013; Campbell, et al., 2014).

Thus, another danger of overemphasizing output versus outcomes and quantity versus quality is creating a system that is a “perversion of natural selection,” which selectively weeds out ethical and altruistic actors, while selecting for academics who are more comfortable and responsive to perverse incentives from the point of entry.

[…]

It is also telling that a new genre of articles termed “quit lit” by the Chronicle of Higher Education has emerged (Chronicle Vitae, 2013–2014), in which successful, altruistic, and public-minded professors give perfectly rational reasons for leaving a profession they once loved—such individuals are easily replaced with new hires who are more comfortable with the current climate. Reasons for leaving range from a saturated job market, lack of autonomy, concerns associated with the very structure of academe (CHE, 2013), and “a perverse incentive structure that maintains the status quo, rewards mediocrity, and discourages potential high-impact interdisciplinary work” (Dunn, 2013).

(Please see the full article, which is open-access, for the full references included above.)

What stood out to me about that paragraph was just how well it resonated with my own experiences of academia.

At this point, I have been in academia for seven years: spending four years as an undergraduate student, one year as a Master’s student, and now two years as a PhD student. During this period, I have undergone a transition that has surprised even myself, because it is that drastic.

I grew up wanting to be a scientist. I loved everything about biology, being that girl who’d gross everyone out by having no fear of frogs or other crawling critters, and who’d need to test any trust claim that came her way, taking no claim at face value without proper investigation. In secondary school, I remember that we had a temporary teacher, who, during a lesson in chemistry, asked the classroom who had the career aim of winning the Nobel Prize. I was the only one to raise my arm, and I remember with what pride I did that. I wanted to be a scientist and to discover things that would change the world.

Starting my undergraduate degree, I soaked everything up like an intellectual sponge. I went over and beyond what was expected of me and I remember that one tutor noted (incredulously) on an essay I’d submitted that “40 references for a 1’500 word essay” was a “bit much”. I begged to disagree, since I had done a lot of reading of the primary literature to synthesise the essay, and I wanted to give all authors their due. Similarly, my emphasis on scientific rigour led a senior scientist to proclaim this as “bordering on the fanatic”.

Nearing the end of my undergraduate degree, having completed five lab-based internships and one final-year research project, something had changed. I’d started to get frustrated with science, having developed an intense dislike for the culture where, just like everyone says, quality had been superseded by quantity. This is not because I had any bad practical experiences, far from it – all the labs I ever interned in were full of lovely people – but because of a strong sense of disappointment that science wasn’t as idealistic as I’d imagined and expected it to be.

I’ve always held scientists in an almost super-human regard. I expected them all to be maddeningly intelligent and resourceful,  and honest, most of all. To me, being a scientist meant something over and beyond everything else; to me, being a scientist meant that you had dedicated your life to the pursuit of truth. It was bit a reality-check for me to realise that scientists are just people – people who don’t worry too much about the cutting of corners and mis-citing the literature and discouraging the publication of negative results. Over time, these realities chipped away at my reverence, and now, another three years down the academic line, I feel unbearably disappointed by the whole enterprise.

As a friend very eloquently put it to me at one point, I am an “idealist about motives”. For scientists, who claim to be after the truth, the cutting of corners proved to be a reality-check that I simply could not bear.

So that paragraph above resonated with me. I am female (so I’m the ‘W’ in ‘WURM’) and I went into science (the ‘S’ in ‘STEM’) with altruistic motives (to serve society). But the reality of the scientific enterprise and its culture has disappointed me to the point where I simply cannot bear it any longer.

I’m tired of reading papers that cite an interesting finding, only for me to look the reference up and find that it says no such thing. I’m tired of reading papers where they say they used line ‘X’ only to find that line ‘X’ is not in the materials and methods, and the entire paper therefore being a big methodological dead end. I’m tired of blase principal investigators and lab heads who worry more about p < 0.05 rather than the overall likelihood that the finding is genuine, or encouraging experiments to be re-done to allow you to obtain the ‘correct’ result for the paper. I’m tired of the kind of model-driven research, where the model is developed before the results are obtained, and the discussions that invariably follow about how the results can be ‘improved’. And I’m tired of scientists willingly giving up the intellectual rights to their work for the ‘privilege’ of publishing in a respected journal. I’m tired of people going ‘ga-ga’ over the prospect of publishing with Science or Nature, rather than the prospect of being rewarded with the feeling of accomplishment that comes with producing quality work.

More broadly, I’m tired of ‘streamlining’ of scientific education in the interest of saving money. I’m tired of universities proud of being rated ‘top-10’ taking active steps to reduce the quality of the education that they offer. I’m tired of institutions that paint themselves as ‘a great place to work’ that keep you without all the essentials of doing that work well.

As Edwards & Roy (2016) say:

The [‘academic excellence’] rankings rely on subjective proprietary formula and algorithms, the original validity of which has since been undermined by Goodhart’s law—universities have attempted to game the system by redistributing resources or investing in areas that the ranking metrics emphasize.

I’m tired of all that – and so many other things. There’s something very rotten at the heart of academia, and of science itself. I’m tired of the fact that people get away with doing all of these things.

I’m currently in the third year of my PhD, and I’m so looking forward to finishing, so I can leave academia behind, to, as the article says “leave STEM to seek careers and work that is more in keeping with [my] values.”*

^{*Although this has put me in the awkward position of trying to change career path with a CV that says nothing but ‘biology-biology-biology’. I might write another blog post on this.}

That’s why the article on maintaining scientific integrity in the 21st century resonated with me.

---

 

Reference:

• Edwards Marc A. & Roy Siddhartha. Academic research in the 21st century: Maintaining scientific integrity in a climate of perverse incentives and hypercompetition. Environmental Engineering Science. September 2016, ahead of print. doi:10.1089/ees.2016.0223.

Are we asking the wrong questions about peer review?

It’s important to keep in mind that scientific publishers are businesses, and money is cccc

I’ve been reading a lot about peer-review lately.

I’ve previously been of the opinion that peer-review is far from a perfect process, and that many (researchers and lay-people alike) are putting undue trust in the system. Arguments in favour of peer review focus on the idea that peer review functions as some sort of ‘quality control’; as a ‘gatekeeper’ or ‘checkpoint’ process that separates the bad research from the good research – of which only the latter is allowed to be set free upon the scientific community to form part of the scientific literature and lay the groundwork for future research.

In a 2013 article on the Laboratory News website, the Vice President of Health and Medical Sciences at Elsevier, Peter Harrison, is quoted as saying:

Publishing in high-impact peer-reviewed journals helps researchers’ careers as it puts a ‘quality stamp’ on their research and helps them get visibility and recognition.

It’s not surprising that scientific publishers endorse the idea of ‘peer review as quality control’. Scientific publishing is, after all, the main provider of the service, and it is the industry that requires peer review the most. Since peer review endows research with a mark of approval of its perceived quality, scientific publishers who wish to appear reputable and as taking their work as guardians of scientific information seriously, must ensure that articles they publish are also peer reviewed.

However, as efforts such as Retraction Watch have shown, peer review is far from a perfect filter, and even the most prestigious and most well-respected journals have failed in their quality control, allowing both since-debunked and fraudulent research to bear their name. However, it is still possible that although peer review is not perfect, that it still manages to keep a lot of sub-quality research away from reputable journals.

Shroter et al. (2008) however report that peer reviewers for the British Medical Journal found very few errors, of both major and minor importance, in papers they had been sent to review – results that suggest that peer reviewers are, on the whole, unable to find all errors in the manuscripts they are sent to review. This finding is not surprising, since peer reviewers are, as the name suggests, peers. Like the authors of manuscripts, reviewing peers are also hard-working researchers, with all the responsibilities that this will bring. It has been estimated that a good peer review effort can take everything from 5-8 hours of a reviewer’s time – which is the equivalent of a large part of a normal work day. It is therefore unlikely that most reviewers, however well-meaning, will be able to catch most errors in the papers they have been sent to review; they simply do not have the time.

So peer review is not perfect, and that is not even going into all the other confounding issues, of author and reviewer bias and deliberate attempts to mislead. This is part of the reason why I have been – and still am – skeptical of peer review: the system is simply not as good as people seem to think it is.

What are the alternatives? Opinions differ. Some argue for a post-publication peer review system, where manuscripts are published first, and reviewed second. I have traditionally been a fan of this system, since it is (under ideal conditions) meritocratic, where only the best papers will prosper. As initiatives like PubPeer have shown, it can also be very efficient at finding questionable conclusions and deliberate attempts to mislead in the scientific literature. Similarly, as physicians’ and mathematicians’ use of arXiv have shown, pre-publication peer review can also be a very constructive way of improving papers and disseminating scientific results prior to formal publication. These kinds of systems however require a culture that is open to them, and I worry that perhaps the life sciences are not.

Nature Publishing Group trialed a pre-publication system of peer review in 2006, but found that enthusiasm was low. While there was sufficient interest for Nature to trail the system, once it was up and running, it failed to deliver as potential reviewers seemed reluctant to review papers, or indeed leave comments to improve the manuscripts, and comments that were left on the manuscripts were of ‘limited use’ according to editors.

In some sense, the results Nature obtained from their experiment make perfect sense. As with the problems with peer review mentioned above, academics are busy. Peer review might work in its current form because manuscripts are sent by the editors to specific authors, who thus are ‘assigned’ the job of reviewing the paper. Being personally given the responsibility to review a paper, a reviewer with some time and/or altruism to spare may be more incentivised to perform a proper review of the paper, especially when the editor is capable of sending them reminders, nudging them to get on with the review. Conversely, in an open system, these direct assignments no longer exist, and all of a sudden the job of reviewing a paper becomes ‘someone else’s job’, and a problem that can be pushed aside for individual, potential reviewers. In an open review system, where anyone can be a potential reviewer, who is going to step up to the task? Very few, would have been my guess, and, indeed, this is what Nature found. When peer review is everyone’s responsibility, it becomes the responsibility of none.

Similarly, I think this is part of the reason why peer review is seen as a method of quality control. Even if there is little evidence of its success in acting in such a capacity, it is a comforting thought. It is easier to open your article of choice — and rest assured that its contents have been peer reviewed, and just assume that everything that it contains is reasonably close to the truth — than to read every article with an unerringly critical eye. It is draining to question everything, especially for long periods of time. In that sense, the idea of peer review as a gatekeeper of truth, may be more comforting than true.

Peer review is not perfect, and, so it seems, neither are the alternatives. So where does that leave us? Where does that leave me? As I mentioned at the beginning of this post, I am traditionally very skeptical of peer review, and I want to believe in the merits of pre- or post-publication efforts to source reviews from a larger pool of peers. It would be great if peer review was a community effort – to the same degree that publishing seems to be.

When I first went into science, this was the kind of idealised view I held; thinking that science was something that scientists did together, where everyone contributed and received their due. With more experience, I now realise how naïve such idealisations are. And I think ideas of peer review are similarly so. Peer review in its current form is not perfect, because scientists are busy, and crowd-sourcing efforts are going to be brought down for the same reason: scientists are busy. It’s easier for a busy person to deal with one paper being added to their to-do list than for a busy person to find the time to contribute to a never-ending pool of scientific results awaiting review.

I’m tempted to think that more honour is needed in science, but this, too, would be unrealistically idealistic. As much as I want to propose a new system of peer review – a perfect one – many such ideas have already been proposed — and they all fall foul of the same assumption: that a good idea is all that it takes to change the culture and the peer review process. Instead, I will conclude with the insight that, having read more about peer review, and the more I know about the process – the more I realise that I don’t know how the problems with peer review should be solved.

But I think that a good place to start would be for the consumers of scientific literature to be educated about the pitfalls of peer review and its imperfections; to make the consumers of scientific literature understand that peer review is not a mark of unerring quality. Second, peer review should worry less about impact, and more about the methodological soundness of the work. This is what PLOS ONE does, but I am also aware that there are mixed feelings about this in the community, with some people worrying that without some kind of editorial predictions of impact and importance, the literature will be flooded with inconsequential papers. However, there are two problems with this view. One, there is no information on how many ‘bad’ papers peer review are keeping away from the community; if you want to publish your work, you’ll always find a journal willing to publish, so this argument doesn’t hold any water. Second, there seems to be some redundancy to the argument of what’s ‘important’, on one hand, peer review is supposed to identify important papers and bring them to attention, whereas, on the other hand, citations are supposed to identify important papers and bring them to the attention of the community. And then there’s the additional consideration of what’s important? Is everything new important? But what about replications? – they are important too. And what one scientist finds ‘boring’ is the topic of choice for another.

In fact, I think my readings about peer review have made me realise that when we speak of peer review, its problems and how to fix these – that we’re actually asking the wrong questions. It’s not the peer review system that is flawed; it’s the publishing system itself. The problems with peer review are the symptom, not the cause. Instead of seeing charities and Research Councils spending money on new journals and new initiatives to improve peer review, what I would like to see is an initiative that’s trying to fix the problems at the bottom of it all. And, the way I see it, journals are part of the problem, since they obsess over ‘importance’ and ‘impact’ and worry more about impact factors than about negative results and attempts to replicate previous findings (or so it seems).

It’s important to keep in mind that scientific publishers are businesses, so we cannot expect them to prioritise the needs of the scientific community over their own monetary bottom line. It has been noted that high-profile journals are in the business of publishing ‘impactful’ papers — even though these are not necessarily the most scientifically rigorous. Flawed papers will also attract citations — and incidentally, increase the impact factor of the journal. For example, the ‘arsenic life’ paper has over 400 citations on Google Scholar, despite having been shown to be erroneous.

Indeed, what I would like to see is a system that gets rid of journals altogether. Imagine, and this is a very crude sketch, that there was a centralised system where manuscripts (results) were deposited, and where they could be retrieved by other researchers since this is what constituted ‘the scientific literature’. Here, there would be an emphasis on results: this is what these researchers found. When other researchers accessed these results, they could not only be prompted to share their view (do these results make sense? Have you replicated these results yourself?) and comments, so that results become less a section-header in stand-alone manuscripts, and more building-blocks in a larger community endeavour, where individual results are shared and used between researchers and research groups.

Because, naïve as I am, I like the idea that science is about sharing, so everyone can learn a little bit more about the world, rather than the idea that science is defined as a process of scientific career advancement — which it seems to be right now.

 

Science is about opening, not closing, doors

In a previous post I wrote that open access is about education. The argument I made in my previous post was tangential; it was the conclusion of a few thoughts I typed up in discussing the arguments made by people who are against open access. So I want to take this opportunity to elaborate a bit on my argument.

Scientists have an interesting role in society. On one hand, they’re seen as being wise and intelligent; and being called ‘an Einstein’ is a compliment. On the other hand, people without personal experience of science seem to subscribe to the popular stereotype of the ‘mad scientist’ to various degrees, and think that scientists are not to be trusted. There is a considerate anti-science movement in the world, with people who claim all sorts of outlandish things that betray a misunderstanding of how science works and what the scientific literature actually says. The anti-science movement is, in other words, based on a misconception of what science is and what it can do, and these misconceptions are giving rise to mistrust.

It is vital to the scientific enterprise that the results are trusted — by both scientists and non-scientists; experts and non-experts alike. But to be trusted, science needs to be communicated. Science is, traditionally, communicated through publication. It follows that good science communication involves easy access to the scientific literature.

Change happens gradually, and the world will not be educated overnight. But for every person who is spreading ideologically-motivated misinformation, there’s another person who wants to check if this is true. And it is for these people that the scientific literature must be accessible, because educating even a single person out of ignorance is, on the whole, a good thing, and exactly what the scientific enterprise is about.

In fact, I’d go so far as to say that science is about more than the scientists themselves. The purpose of the doing of science is not to facilitate scientists to make a career. No, rather, science is about expanding the body of human knowledge — and not just for science; we do this for humankind. And seen this way, it seems immoral to put whatever we find behind lock and key. To do so is to act selfishly; jealously guarding whatever we find because it can benefit not everyone, but ourselves.

Going back to my previous post, which came about after I read Mike Taylor’s post on the bizarre comments made by some people who don’t believe in open access, it seems like argument from personal gain is a recurrent theme: that it’s a ‘fallacy’ that ‘non-experts should read journals’. Indeed, this is even further elaborated by Robin Osborne at the Guardian, where he makes the argument that open access ‘makes no sense’ because access comes at a price. And that price, he argues, is the admission to the institutions where research is being carried out:

For those who wish to have access, there is an admission cost: they must invest in the education prerequisite to enable them to understand the language used.

But this kind of education can come in many forms. And one of those is free access to the literature. You don’t need to be an expert to make sense of things. It helps, but it’s not a prerequisite. Furthermore, this means that there is a parallel argument to be made: that research should be freely accessible both intellectually and financially. You shouldn’t have to pay to access research, and that research should be well-written. Indeed, there is an argument to be made to reduce the amount of jargon used in scientific papers, because what is jargon except another barrier that is used by those ‘in the know’ to keep those who are not in the dark?

And that brings me to my second observation, that a lot of the hostility that is directed against open access seems to stem from a sense of academic insecurity; that science is only worth something if it is ‘pure’. Osborne says:

There can be no such thing as free access to academic research. Academic research is not something to which free access is possible. Academic research is a process – a process which universities teach (at a fee). Like it or not, the primary beneficiary of research funding is the researcher, who has managed to deepen their understanding by working on a particular dataset. The publications that result from the research project are only trivially a result of the research funding, they come out of a whole history of human interactions that are not for sale.

It seems like Osborn is saying that open access is impossible because it would threaten the ivory tower of academia and that, as a result, it would dilute the academic enterprise. It’s exceedingly elitist to suggest that the uneducated don’t deserve access to the scientific literature, as this would reduce its value, but, alas, it is also a very common argument.

It’s very interesting that this argument is common. It seems to imply that academics and researchers and scientists who are against open access oppose it on the grounds that they think their role will be lessened if the ivory tower was to open its gates; that the value of science is inversely proportional to the ease by which it can be accessed. This is not true. Rather, it is the opposite; the value of research increases the more it is accessed. Indeed, isn’t a high download rate and high citation count what every researcher yearns for — since these metrics measure the amount of times information has been disseminated and impact has been made?

I don’t want to read too much into this, but part of me wonders if at least some hostility towards the idea of open access comes from the misconception that the status of being a scientist comes not so much from being a facilitator of the spread of knowledge as much as a custodian of the unknown?

What boxes that are not boxes can tell us about ourselves

The Ashmolean Museum here in Oxford currently has a special exhibition on Andy Warhol. I’m not usually a fan of modern art, but the exhibit was free for students, and a friend therefore managed to coax me into spending a Saturday afternoon there.

Andy Warhol at the Ashmolean.

The exhibit featured works from the Hall collection, and as far as I (in my Warhol-naïvety) was aware, there were none of the typically iconic pieces on display. However, I found the exhibit surprisingly enjoyable. There was a good selection of pieces, most of them colourful. The exhibit was divided into four sections: one showing some of his less well-known and more experimental pieces; another showing some of his films; a third showing the characteristic pop-art portraits; and a fourth showing some of his black-and-white work.

I’ve never understood modern art, other than that the artist is trying to provoke. Although the definition of ‘art’ is nebulous, I think it’s valid to differentiate between art that tries to portray something (which is what most people would call ‘art’), and art that tries to provoke (which I guess would be classified as ‘modern art’).

However, while I was at the exhibition, I saw this display case that contained a single Brillo box. The box wasn’t as much a cardboard Brillo box as a wooden crate that had been painted to, for all intents and purposes, resemble an authentic Brillo box.

Brillo boxes via Richard Winchell.

The exhibition was quite busy, so the box in its case was quite busily looked upon. While watching people looking at the Brillo box and trying to make sense of it, something occurred to me: that maybe that was what the box was about. That maybe the Brillo box wasn’t meant to be looked at directly — but that it was meant to be looked at by some people, so other people can look at people looking at it! Once I realised this, the box made so much more sense.

Perhaps the provocation of modern art isn’t so much as to provoke you personally, it’s about creating a provocation that you can then observe; and that this is what is the art. Or perhaps not so much ‘art’ as the statement. Because most modern art is statement-driven; having been created to tell you something. Which is perfectly valid: because humans are empathic creatures; we must feel things ourselves in order to truly understand those things. And maybe that’s modern art: it’s a form of non-verbal communication that allows us to experience something in order for that experience to be properly communicated and with nothing lost in translation.

So perhaps the Brillo box isn’t so much about being a Brillo box or an artist’s impression of a Brillo box, but perhaps it’s meant to be an artist’s way of communicating the absurdity of culture; of perception; of existence. Because it is rather absurd to have a Brillo box that isn’t really a Brillo box, but just a painted wooden crate, set in glass. Even more absurd is observing people looking at this not-quite-a-Brillo-box, as if it was anything more than it is. Because I’ve noticed that people do have a tendency to read more into provocative pices like that than is justified. Sometimes a box is just a box (or not even that). But since we’re social creatures, and ones terrified of not getting something, of not understanding, of looking stupid, we can look  at a box and think to ourselves that it’s just a box and then second-guess ourselves and think that we’re not being intelligent enough, not cultivated enough, not sophisticated enough, and pretend that, we too can see the Emperor’s new clothes. And while all of this is going on, a separate observer, watching this person experiencing all the second-guessing and pretending that’s part of being a human in a social/cultural situation, is given a unique glimpse into the human psyche, allowing the inner world of someone else to be laid bare for us to experience — and therefore understand.

Once I’d had this revelation, modern art made s much more sense to me. And all thanks to a Brillo box that’s not actually a Brillo box.

–//–

 

As an aside, here’s a funny story: I was once standing right in front of a set of Warhol’s Marilyn Monroe-paintings, which are arguably his most iconic. I didn’t think twice about them, because they are so widely reproduced that they have become commonplace. It was only later that someone else pointed out to me that the pieces were authentic.

Marilyn via Ian Burt.

And that set me off thinking: why do we place such importance on authentic pieces of art and not reproductions? What’s so special about the original that we stop in our tracks to take it in, or even travel to a particular place to see it, in the metaphorical flesh? Because the experience of looking at a replica (at least a decent one) isn’t particularly different. It’s the same shapes, the same colours, the same everything. Except that it’s lacking the special ‘it’ of once having been in touch with the creator that once gave rise to it.

The answer to this discrepancy, I think, lies with our cultural obsession with talismans and similar objects, of thinking that the original is imbued with something immaterial and special that replicas just cannot reproduce. What do you think?

Scholarly access is about education

I read Mike Taylor’s post on how Sci-Hub is a ‘litmus test’ a few days ago. The post highlights some of the people who are sympathetic to Sci-Hub, as well as some of the people who are hostile towards the initiative. Reading the post, what stood out to me was the inanity of some of the hostile comments — some of them making the most bizarre claims.

Among the more bizarre claims are the ones to the effect that keeping the scientific literature locked away is a good thing. Taylor quotes David Wojick who writes:

I personally doubt that there are large numbers of people who (1) have the expert knowledge required to read and benefit from the scholarly literature but who (2) cannot find a way to access what they need. The arguments I have seen to this effect are completely unconvincing.
[…]
This is one of the fundamental fallacies of OA, namely that non-experts should read journals. […] Only a few people can understand the typical journal article.

That’s a very elitist argument, and for two reasons.

First, these blessed people Wojick speaks about who don’t have problems with access don’t exist. Anecdotally, but not irrelevant to the issue at hand, I’m currently doing a DPhil at the University of Oxford — supposedly one of the most prestigious universities in the UK. Yet, it is a regular occurrence that I find a paper that I think looks interesting from the abstract, but that I find, to my dismay, that I can’t access because the Bodleian Library doesn’t seem to have a subscription that covers that particular journal. If there is no freely available copy easily accessible via Google Scholar, then what is a poor, suddenly-not-so-privileged DPhil student to do?

Second, to claim that it’s a “fallacy” that non-experts would want to read scientific journals is beyond ignorant. Not only is that a moot point that does nothing to support Wojick’s argument — in no way should the lack of desire for access mean that access should be denied — it’s also making a very unpalatable claim, namely, that people who aren’t experts have no business reading the expert literature. That’s like saying that people who don’t have an education don’t deserve one. That is just silly.

In addition, there is a third argument, and one that is very close to my own heart. Even if I don’t have access to all the papers that I need/want for my own research, I still have access to a decent amount of the scholarly literature. Because of this, I regularly download papers and articles on behalf of friends and family. I do so for friends who don’t have access via their own institutions, or for friends who recently finished their degrees, which leaves them without an institution, and therefore without access. In such scenarios, the system is preventing access to legitimate experts, which completely invalidates Wojick’s claim.

In addition, although I’m only half-way through my doctorate, I’m already thinking about life after I’ve completed my degree. It’s been a long while brewing, but I do not intend on staying on in academia as an active researcher. The reasons for this are many, and will form the topic of a future post, since they aren’t relevant to the current issue. What is relevant is the fact that there will be a day when I no longer belong to an institution; that there will be a day when I no longer have access to the scientific literature. And this saddens me immensely. Because over the almost seven years (and counting!) that I’ve spent as a student of science, I’ve become increasingly reliant on the scientific literature. If I want to educate myself on a topic, it is to the scientific literature that I turn. To be denied access because leaving science makes me ‘unworthy’ is immensely discouraging. And this brings me to an even more fundamental issue:

Just like I trust in the scientific literature to educate me out of my ignorance, so do other people. Science is trusted by society. Experts and non-experts trust science to be a force of enlightenment. So we should therefore not be surprised that they want to access the scientific literature itself. This is a very important consideration, because we cannot ask of the public to trust in science and then deny them access because they aren’t ‘worthy’. That runs counter to the scientific enterprise as a catalyst of knowledge. So anyone who’s pro-science and pro-education should also be pro-access. It’s as simple as that.

No Precedent

Thinking back to the beginning of humankind, what a strange event it must have been to ‘awake’ as a conscious creature. Just imagine, what a strange thing for an organic being of tissue and bone to consider its own purpose — for the first time. At the beginning of culture, there must have been discussions of how best to do things — how do you even do things, when there is no historical precedent? Imagine the first funeral (what to do with the dead?); the first painting (why depict anything at all?); the first idea of a god (why was there such a need to seek explanation, when other animals need none?).

Last Sunday of September

It’s the last Sunday of September. I don’t know where all the time has gone! It seemed like summer was stretching forward, indefinitely, with promises of adventures yet to come…! It is always somewhat forlornly that one thinks back of all those beautiful summer eves that were spent doing nothing at all, since there was another day just another peaceful sleep away.

September is when Autumn inches forward and somehow, every year, manages to catch you by surprise. This year’s September has been wonderfully sunny, and this weekend was no exception. Having lingered around most of the morning, and having had a traditional Swedish pancake lunch (fried in butter until golden, and served with a sprinkle of sugar on top), I ventured out into the sunshine.

Following winding footpaths across fields and through villages, I found myself full of energy — making a brisk September countryside walk the perfect end to the week.

The local farm had some gorgeous sunflowers shining in the afternoon sun, and I noticed that the walnut trees in the background were full of fruit. I’ve been living in the UK for six years, to the week,  but I’m still childishly fascinated by the ‘exotic’ trees and plants that thrive in English soil.

My walk reached its end in a local village, where I had a look around before turning back. Walking around the not-yet familiar streets and pedestrian footpaths, I had a feeling of having arrived somewhere, even if it wasn’t the plan. Sometimes such things just happen, with no foresight to guide the way. To me, it just added to the feeling of adventure.

The local church had organ music streaming through it’s open doors, adding to a feeling of familiarity. Village people were comings or an evening prayer, tracing the path from their own dwellings onto that of the church.  Though not religious myself, I do appreciate the sense of community that a shared ritual such as Holy Day worship must bring. To me, organ music brings back memories of the last day of school, and all the excitement of freedom soon to be had.

I must remember for next year to make better use of the summer evening light, for it is gone so fast. September is when Autumn inches in on you,when days grow shorter, and when evening shadows grow longer, no matter how fast you walk.

Heading homewards, I brushed against blackberries, regretting not having brought a bag to collect them in. I remember an evening with a fiend, when I filled my hands with foraged berries till I could hold no more,, and returned home to make delicious crumble. Instead, I listened to chickens cooing, and admired the gilded hills in the sunset distance, all the while wondering what’s meant to be done with all the sloes.

It is very quiet over the fields in the early evening, although the cry of a gull from the calm lake can be heard over the hedgerows. Getting closer, I could see rabbits bounding, bright cottontails in the grass. A fox ran past, chasing them, but more like a farmer shoos his sheep than a predator stalking its prey. A few foraged apples that had found their way into my pockets found a purpose as a stranger’s gift to a glossy, brown horse.