It has been nearly two decades since The BMJ introduced “Rapid Responses” with HighWire, and nearly as long since Pediatrics introduced “Post Publication Peer Review” (“P3R”). These were the two earliest examples of the use of HighWire’s “eletters” technology. Unlike some other innovations of that time – e.g., “download to PowerPoint” – there were few followers to these leaders in online commenting. Even after Tim O’Reilly declared that Web 2.0 was upon us – when the readers on the web became writers, and a conversation ensued – only a few HighWire-hosted journals picked up commenting features. Commenting seems everywhere on the web these days but scholarly journals.
Where did the conversation go, and why isn’t it happening in journal sites?
The Big Start: The BMJ’s Rapid Responses
The BMJ had a very active Letters to the Editor section prior to launching the eletters feature. So The BMJ’s readers already were having journal-mediated conversations before the web showed up. But when eletters began, the cycle time for letters was reduced from weeks to hours. From the beginning, The BMJ’s policy on moderation of comments was inclusive: if a submitted eletter was not libelous, obscene, or self-serving, it was published. Subsequently, The BMJ added criteria to reduce the pile-on “me too” kind of behavior, and the trolling: posts could not be boorish, and had to advance the dialog.
The result is that in seventeen years of eletters, The BMJ has posted as many eletters as it has published articles. That’s as many eletters in seventeen years, as articles in 175 years! The BMJ is by far the most active eletters venue among HighWire-hosted scholarly publishers, publishing from 300-600 letters/month. I suspect it is the most active of all scholarly journals. (If you know of others more active, please comment.) The letter writers are predominantly physicians, about evenly split between primary and secondary care.
David Payne, web editor of The BMJ says, “My sense is that it’s very driven by content, so when we publish on controversial topics (such as assisted dying, our most recent example), we see lots of engagement.”
Blood differentiates between letters as commentaries on articles, and letters that present new information. The latter are more appropriately handled as short-form articles, which get DOIs, and can be discovered, indexed, cited, etc.
Pediatrics reported they receive 12-15 comments monthly, and hope to see this grow through use of a new interface that guides users to content in their specialties.
PNAS notes the articles that have the most comments are “the controversial or more highly publicized articles… We have 40+ on the Facebook article, and several on the Bruce Alberts et al. piece about the biomedical industry.”
BioRxiv is particularly interesting, because the comments are made on preprints that are posted, not on articles that are published. Richard Sever, Assistant Director at Cold Spring Harbor Laboratory Press, said,”bioRxiv has more commenting than our journals, certainly. I think our most recent estimate was that something like 10% of bioRxiv articles have comments. My impression is that when there are comments at all, they tend to cluster. Papers in evolutionary biology, particularly on human ancestry, tend to attract multiple comments: that seems to be a field where there are rather a lot of theories, which I think always drives discussion. And these people seem to be very interactive. In other fields, not so much.”
John Inglis, Director of Cold Spring Harbor Laboratory Press, commented further, “I think some of the responses underline the fact that a certain subset of scientists are intrinsically interested in the process of communicating science. This subset of scientists responds to work that analyzes or investigates the communication process and they use the tools of communication to do so. So in the subset, there is a greater proportion of bloggers, tweeters, and people likely to take advantage of a commenting function. The subset is also skewed by scientific interest: more people from genomics, bioinformatics, and population genetics, fewer from cancer research or physiology. The use of preprints is another of those activities that interests the hyper-communicating subset.”
The Barriers – What Readers Think About Commenting
In HighWire’s interviews with researchers we found little interest in reading or writing commentary on published articles. Anecdotally, this did not appear to be generational. Among readers there was a bias against “vox populi” comments, though there was an interest in “informed commentary”. This suggests that researchers are interested in comments from reviewers, co-authors, and experts on a topic, but are not willing to sift through student questions, or non-experts’ misinterpretations.
Reddit is very active in carrying commentary about scientific topics, and about articles. But it is a mixed bag of informed (but usually ad hoc) comments, and others.
The Barriers – What Writers Think About Commenting
If the scholarly process is a type of dialog leading incrementally toward the truth, then the way scholars converse is to write papers, read papers, and write papers in response. So you could say that the time-honored way to debate a paper is to write a paper in reply, with the usual evidence a paper has: data, references, etc. In fact, you can see some (few) eletter writers following this model by including figures and references in an eletter.
So why don’t we see more eletters with figures and references? A number of people have observed that the incentives and rewards for writing substantial (i.e., evidence-based) eletters simply don’t exist. In fact, there is a disincentive – especially among younger researchers – to spend research time on something that doesn’t lead to publication credit or some type of recognition. Eletters are a type of grey literature: they are not formally published, often don’t have their own URL, aren’t given a DOI, aren’t indexed, etc. In short, you can’t put an eletter on a CV.
One additional challenge for scholarly eletters is worth noting: the process of reading and responding doesn’t fit the “news cycle” that feeds further commenting. On the general web, commenting is conversational back and forth, like ping pong or volleyball. But commenting in a scholarly context is perhaps more like chess: the quality of the result is probably inversely related to the speed of play. Yet the conversational model requires a kind of temporal continuity that comes from replies that take place in the fresh-news cycle. If you read an article, consider it, draft a comment, submit it, and it gets posted well after the next issue comes out, then the eletter is really no different from the print Letter to the Editor cycle.
Good responses take some time to add evidence to ‘mere’ opinion. The time it takes to carefully formulate a response – which could require some literature research – inhibits the conversational model: by the time you speak up, the author has, metaphorically, “left the room”. By the time I’ve written and posted a cogent response and had it moderated, the readers and the authors of a piece have moved on to other work. The delay cools – if not kills – the spark that ignites dialog into conversation.