Write to Cite: Writing Style and Citation Counts

A pair of articles bookended the summer with bibliometric data that tells us something about the correlation of writing style and citation counts:

Shorter-title articles have more citations.(1)

Longer-abstract articles have more citations (2)

At first these seemed contradictory to me. Surely either less is more, or more is more? I read the longer-abstracts piece (not an article, but an editorial that includes analysis) early in the summer and imagined a logical progression:

  • more words in the abstract would match
    • more search engine queries and more matches would lead to
      • more presence in search results which would lead to
        • more readers, which would lead to
          • more citations.

Indeed, the authors of the “articles with long abstracts have more citations” essay describe something like this as a hypothesis:

“We find that shorter abstracts (fewer words [R1a] and fewer sentences [R1b]) consistently lead to fewer citations, with short sentences (R2) being beneficial only in Mathematics and Physics. Similarly, using more (rather than fewer) adjectives and adverbs is beneficial (R5). Also, writing an abstract with fewer common (R3a) or easy (R3b) words results in more citations.” (2)

And

“Despite the fact that anybody in their right mind would prefer to read short, simple, and well-written prose with few abstruse terms, when building an argument and writing a paper, the limiting step is the ability to find the right article. For this, scientists rely heavily on search techniques, especially search engines, where longer and more specific abstracts are favored. Longer, more detailed, prolix prose is simply more available for search. This likely explains our results, and suggests the new landscape of linguistic fitness in 21st century science.” (2)

But then there’s that “articles-with-short-titles-have-more-citations” study, published in late August. (1) I couldn’t come up with a logical progression that seemed to indicate why short titles would correlate with more citations. The opposite came to mind, in fact: in researcher interviews in 2013 and 2014, HighWire heard a preference among readers for article titles that were like a declarative sentence: “A catalyzes B in the presence of C.” These would not tend to be short titles. Many publishers have begun including such statements on their tables of contents, and researchers tell us they like this. HighWire calls these “annotated TOCs”; Science for example does these very well in its TOC. But the ‘annotations’ are typically longer than the authors’ article title.

(It was unfortunate, perhaps, that these two articles were completely complementary: one looked only at titles, the other looked only at abstracts.)

Should editors counsel authors to shrink titles and expand abstracts? Neither study investigated or demonstrated causation. That is, shrinking a title or bulking up an abstract isn’t demonstrated to increase (cause) citations, only that short titles and long abstracts are associated with or attributes of (correlated with) more highly-cited articles.

Neither article demonstrates an explanation for its findings, though each has a hypothesis. But the hypothesis about titles doesn’t work for abstracts, and vice versa. Here’s the explanation offered in the “short-titles-more-cites” paper:

“We propose three possible explanations for these results. One potential explanation is that high-impact journals might restrict the length of their papers’ titles. Similarly, incremental research might be published under longer titles in less prestigious journals. A third possible explanation is that shorter titles may be easier to understand, enabling wider readership and increasing the influence of a paper.”

The complementary long-abstracts-more-cites article looks at 15 guidelines commonly recommended to writers of scientific articles.   It provides evidence that eight of the rules are wrong-headed, correlating negatively (in red) with citations:

journal.pcbi.1004205.g001“Fig 1. Effect of abstract features on citations. For each discipline (rows) and each abstract feature (columns), we measured whether a certain feature (e.g., having fewer words than the typical abstract published in the same journal [R1a]) led to a significant increase (blue) or decrease (red) in total citations. We considered an effect positive or negative only if the associated probability of being zero was smaller than 0.01/15 (i.e., we applied the Bonferroni correction to obtain an overall significance level of 1%). doi:10.1371/journal.pcbi.1004205.g001”

The short-titles-more-cites paper has a more complicated story to tell (despite its shorter title): the negative relationship between title length and citation count is clear for articles 2007-2011, but survives only at the journal level for 2012 and 2013. The authors summarize thus:

“Our analysis suggests that papers with shorter titles do receive greater numbers of citations. However, it is well known that papers published in certain journals attract more citations than papers published in others. When citation counts are adjusted for the journal in which the paper is published, we find that the strength of the evidence for the relationship between title length and citations received is reduced. Our results do however reveal that journals which publish papers with shorter titles tend to receive more citations per paper.”

What to make of this?

Researchers tell us they find their way to articles using several routes:

  1. Personal recommendations
  2. Email tables of contents (for favorite or “followed” journals)
  3. Scholarly search engines (e.g., Google Scholar, PubMed, Scopus, SSRN)
  4. General search engines (e.g., Google Web Search)

Article titles are a primary UI for #2,3,4 in that a title is what shows up, plus matching text shows up in “snippets” for some of #3,4.   But for search engines such as Google Scholar, matching a search query is not based solely on title or abstract, but on full text. So abstract words are not primarily there to match query terms in the case of search engines that index full text, like Google Scholar.

But those abstracts are important for the filtering steps that a researcher uses to decide whether to invest more time in reading.   My colleague Anurag Acharya, who leads the Google Scholar team, offers this explanation:

“Researcher workflow … is often structured as multiple filtering steps — do a query, scan results list and pick some abstracts to read, read these abstracts, pick some fulltext articles to read. Longer/more detailed abstracts have the potential to help the paper make it through the second filtering step (read abstract -> pick fulltext).”

But what to make of the correlation with short titles? Sometimes a shorter title (assuming it isn’t fanciful with low “information scent”) can suggest something more comprehensive, and thus attract the reader who is looking for a foundational article to read – what Anurag calls the ‘name’ paper — and possibly to cite. Consider two titles: “Arabidopsis Mutagenesis” vs. “Genome-wide insertional mutagenesis of Arabidopsis thaliana.”

Think of the research-article equivalent of Wikipedia titles; or the titles in Annual Reviews articles, which are comprehensive pieces that are hugely cited. Anurag had a personal experience with this:

“As it happens, many years ago, I had submitted a paper on disk architectures titled “Active DIsks”. The paper was accepted but the program committee chair insisted I change the title to a longer one. So we went with “Active disks: Programming model, algorithms and evaluation”. . So that the paper didn’t become the “name” paper for this architecture.”

(1) :Letchford A, Moat HS, Preis T. 2015 The advantage of short paper titles. R. Soc. open sci.2: 150266. http://dx.doi.org/10.1098/rsos.150266

(2) Weinberger CJ, Evans JA, Allesina S (2015) Ten Simple (Empirical) Rules for Writing Science. PLoS Comput Biol 11(4): e1004205. http://10.1371/journal.pcbi.1004205

One thought on “Write to Cite: Writing Style and Citation Counts

  1. Pingback: What’s the Definition of “Read”? | HighWire

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s