More on E-Reading – the Academic Literature

In a previous article, I summarized the pros and cons of e-reading in 2020. One area that deserves a little deeper discussion is the scientific literature. I’ve read many articles reporting that e-reading is scientifically established to be inferior to print for comprehension and deep, immersive reading. This would be deeply worrisome if true. But I’ve had a look at the actual science, and I don’t see anything to worry about. So here’s my review of that literature.

Ground rules

I’m only concerned with long form prose – ebooks, essays, novellas, short stories, etc. Not popcorn reading you skim to kill time, not snippets of text you glance at on your cell phone, not articles you consult briefly at your desk just to get the gist. This is about immersed reading to which you devote dedicated reading time.

The scientific literature I care about consists of peer-reviewed studies that present novel experimental data or analysis and are published in academic journals. I’m willing to consider reports in other forms, as long as original scientific evidence or analysis is presented, not just anecdotes and opinions. I’m also willing to consider arguments made by people familiar with the literature, as in one non-academic book I’ve read, because I think the perspective can be informative. But ultimately, perspective is not evidence.

And lastly, I want to emphasize that although I work for an ebook seller, the views expressed here are strictly my own. See the disclaimer at the bottom of this article.

High-level Summary

In my view, there’s no compelling reason to believe that e-reading is any worse than reading in print, as long as you avoid some obvious pitfalls. But the relevant scientific studies are few, and often run afoul of those pitfalls, creating the appearance of a general disadvantage for e-reading.

Although each study has its own strengths and weaknesses, I’m going to focus on two common issues: poor technology and inexperience with e-reading.

Poor technology

What counts as e-reading? The rapid evolution of technology over the past decade poses a fundamental problem for research in this area. Although the scientific literature on “reading on screens” goes back decades, the obvious inadequacy of reading on a fuzzy CRT from the 1980s is a not a reason to avoid ebooks in 2020. Neither, for many of us, are the limitations of devices from 2010.

For practical purposes, we can date the birth of widespread ebooks to the release of the first Kindle in November 2007. E-reading took another step forward when tablets gained traction, with the release of the first iPad in April 2010. Book-sized Android devices also became widely available in 2010, and the Kindle app was released in that year. The iPad Mini, a 7.9 inch iOS device, was released in November 2012. And the devices have improved steadily over the past decade.

The rapid evolution of technology makes it tricky to evaluate the science. If you go too far back, you include heaps of irrelevant data. But if you draw the line too recently, there are very few studies. The problem is exacerbated by the delay from study design to publication (often years) and by the fact that researchers don’t always use the most recent technology. A study published in 2015 may still be using first-generation e-readers. The best recent meta-analysis I’m aware of (Clinton, 2019) excludes articles prior to 2008, but still includes studies with alarmingly poor e-reading conditions. One of the best recent studies I’ve read uses a reader last manufactured in 2012 (and that didn’t get great reviews even then).

Let’s consider two aspects of the technology: screens and software.

Screens. Since 2007, E-ink has improved greatly in contrast, resolution, and refresh speed. LCDs (I lump OLED into this group) have also improved tremendously in resolution, brightness, and viewing angles. Poor screen technology may affect e-reading adversely, so we should be skeptical of results from studies conducted with early e-readers.

Many studies are vague about the display technology used in the e-reading conditions, making them difficult to assess. For example, even just within LCDs, there are very important differences between passive matrix, TN, IPS, and OLED. Specific devices vary in screen size and resolution, contrast, type of illumination, and portability. But many e-reading researchers are either unaware of these distinctions or don’t consider them worth reporting.

Software. The flowing text experience you get from an ebook reader has been designed to support comfortable, immersed reading. However, this is not always what researchers use in their studies. Some studies use desktop PDF readers. But PDF is a format for encoding page images that are normally optimized for printing, and ill-suited for reading on-screen. Some studies use Microsoft Word, which is similarly a poor choice for reading. Others use web pages, or custom software designed for the study, which can be tricky to assess. For many of these studies, the e-reading conditions are remarkably poor. No one conversant with electronic media should be even mildly surprised that PDF readers and Microsoft Word provide a poor reading experience.

If I had to guess, I’d imagine that most handheld e-readers and tablets, even those manufactured as long as a decade ago, are adequate, but not ideal, for immersive reading. To be safer, I’d prefer to draw the line around 2015, when E-ink was much more mature and IPS tablets were more readily available. But many of the studies I’ve read didn’t even use handheld devices, or are nonspecific about what they did use, either of which is a huge red flag.

As a society, we’ve had centuries of practice with print, and little more than a decade with ebooks. So while 10-year old technology is perfectly fine for print, it’s not a fair way to judge the technology. If you’re trying to decide about reading on a 2020 Kindle or tablet, studies using the devices from 10+ years ago are of limited relevance. And while there may be legitimate reasons to wonder about comprehension under a variety of software conditions, neither Microsoft Word nor Adobe Acrobat, nor really anything presented on non-portable devices, is not how people who care about reading settle in to consume long form content. If that were the pinnacle of the e-reading experience, I would be firmly committed to print. We all would.

None of this invalidates the research. But it does suggest that headlines like “Why reading paper books is better for your mind” (Washington Post) may be misleading. A more honest headline would be “E-reading under poor conditions can be slightly worse than print.”

Experience with E-reading

Virtually everyone who can read today learned to do so in print. The typical participant in these studies (a college or high school student) has read hundreds of books in print, and shows a strong preference for print when doing long form reading.

Much of the scientific literature either implicitly or explicitly assumes that young readers, as “digital natives,” will be fluent in reading on screens. But this is an error. While the typical student has plenty of experience with reading on screens in some form, they generally have little or no experience with reading ebooks on a dedicated reading device. They may be fluent in text messaging and social media, but the same is not generally true for long form e-reading. Writers critical of e-reading are often expansive about the distinction between superficial and deep reading, so it behooves us to take that distinction seriously. Endless hours with your math e-textbook open on a Chromebook is plainly not relevant experience for reading Middlemarch on a Kindle.

Only a few of the scientific studies I’ve seen give participants meaningful experience with e-reading (I dive into one such study below). The vast majority are content to present results comparing, on average, experienced print readers with novice e-readers. So again, we can imagine a more honest (but less dramatic) headline: “E-reading can be slightly worse than print if you’ve never done it before.”

Does this really matter? Should we expect a period of adjustment when you begin to consume ebooks? I had a solid 30+ years of consuming online content under my belt before I read my first ebook. Still, I was very conscious of having to adjust when I read my first ebook about ten years ago. I had to get comfortable with the device’s controls, experiment with its font and layout options, and get used to its peculiar rhythm of turning pages. And I was definitely distracted by the novelty of my first dedicated e-reader. Even though I’m a bit of a technophile, it took a book or two for me to feel comfortable. My wife, by contrast, tells me that she felt comfortable with her very first ebook, which she read on a Kindle.

I don’t know how much this contributes to the reported comprehension gap. But neither do the authors of the studies I’ve seen. Given the preference for print (reported in many studies), it’s entirely plausible that a comprehension gap would be caused by the dissonance of being forced to read in a medium you wouldn’t choose. But if so, it doesn’t mean ebooks are inherently worse for those of us who have no such preference.

As with the technology issues, this doesn’t invalidate the studies, but it does suggest a lack of relevance. If you have thousands of hours of reading ahead of you in life, and no strong aversion to e-reading, you don’t want to give up on all the advantages just because the first hour or so will be slightly awkward.

Summarizing

Every study has its strengths and weaknesses, but a plausible interpretation of the findings I’ve seen is that they’re mostly driven by three things:

• Poor (by modern standards) e-reading hardware and/or software.
• Participants who have little or no experience with reading ebooks.
• Participants who dislike long form reading on devices.

These factors may or may not be scientifically interesting, but they’re completely irrelevant if you’re trying to decide how to read. Outside of the context of a scientific study, you can:

• Use a much better device and much better software than the average scientific study (budget permitting).
• Gain fluency with e-reading fairly quickly. In just a few days, you can get ten times more experience than participants in any of the studies I’ve read.
• Decide for yourself if you find e-reading aversive.

One item missing from the above discussion is distraction. Although ebook critics, including scientists, often speculate that it’s a factor, It’s not addressed by the studies I reviewed.

Although I imagine there’s some literature on this that I just haven’t found, the idea that e-reading is necessarily more distracting than print reading feels like lazy thinking to me. It’s not hard to configure modern devices to minimize distractions. I basically never do anything but read on my tablet, and E-ink readers (like the Kindle) are essentially useless for anything else. Certainly it would be hard to explain why, if my tablet is so distracting to me, I have not yet, after a decade of e-reading, actually succumbed to that distraction.

It’s also hard to explain why these distractions are more salient just because you’re reading an ebook. Cell phones can clearly be distracting – they may ring and vibrate and tempt you with email and games. But they will do this even if you’re reading a print book. I’ve seen reports that merely having a cell phone in the room can be distracting, even if the phone is turned off. So perhaps neither print nor e-reading is great for anyone who owns a cell phone. But I doubt that. My phone basically never rings, buzzes, or interrupts my reading in any way. It’s not my first choice for reading (due to the small screen), but I can read surprisingly comfortably if I’m caught away from my tablet.

My experience is absolutely not universal. If you find your reading device inherently distracting, then perhaps print would be better for you.

Could the case against e-reading be worse than we’re told?

Notwithstanding my concerns about the relevance of the existing studies, it’s still possible that they underestimate the harm of e-reading.

If you look at comprehension measures across studies, many of those that report “no significant difference” show a non-significant advantage for print, suggesting the likelihood of a pervasive effect that some studies are just too under-powered to detect. Lack of evidence for a difference is not the same as evidence for a lack of difference. It’s possible that many studies showing similar performance for print and e-reading just aren’t adequately sensitive to subtler differences in deep, immersive reading.

If this is true, then future studies will be needed to work it out. But the evidence so far seems to me more suggestive of a different pattern.

Deep dive into a single study

The criticisms I’ve outlined here apply broadly to the literature, but there may still be individual studies that are more compelling. I’m going to go into a little more depth here with one recent study that I think does a better job than most.

Mangen et al. (2019) avoids many of the pitfalls in study design and reporting. The authors gave 50 participants a long reading sample (28 pages, over 10,000 words) to read over the course of about an hour. Half of the participants read in print, while half read on a Kindle DX, an early Kindle model with a 9.7-inch E-ink screen. Unlike many researchers, they were careful to control things like page size, spacing, and font size – basically, readers in the two conditions saw the same pages, just presented in different media.

They assessed things like reading speed, content recognition, “transportation and engagement,” and five types of content recall. On the majority of these measures, they found no significant differences between e-reading and print. But they did find reliable differences in three measures of temporal order processing. In one such measure, participants were asked to recall whether something occurred in the first third, middle third, or last third of the text. Participants using e-readers fared more poorly than those using print when the event occurred in the first third of the text. In another measure, subjects were asked to place events from the text in order. And again, participants using e-readers performed significantly worse than print readers.

Mangen at al. suggest that these differences in memory for temporal order may be related to “the sensorimotor assessment of the device.” Their explanation is a little hard to follow, but roughly boils down to the fact that materials in print offer more cues about where you are in a book than most ebooks do.

From my perspective, it feels like this finding is just an artifact of the study design. I’ll break down a few issues.

Participants in this study were generally unfamiliar with e-reading – the average response to a question about familiarity with Kindle-like readers was closer to “never used” than to “occasionally used.” This means fewer than half had ever used a Kindle or anything like it. The use of a longer passage gave participants a chance to acclimate to the device during the study, but most were using a Kindle for the first time. Note that E-ink screens are very different from the smartphone and tablet screens familiar to most of us today.

The difference they found in one of their measures, called “Where in the text,” shows an interesting pattern. The statistically reliable difference is present only for events from the first third of the text. Eyeballing the numerical results, it appears that the difference was greatest for the first third, smaller for the second third, and near zero for the final segment. Supposing this pattern to be real, why does the effect wane? One possibility is that, since more recently read text presents an easier challenge, any difference in temporal order memory may have been wiped out by a ceiling effect. Another is that the advantage of print goes away after less than an hour of practice. Although I would like to believe it’s the practice effect, my intuition is that it’s mostly the ceiling effect – i.e., that even with an hour of practice, there is still a real difference in temporal order memory that just didn’t come out in this study. But it’s possible that any differences in temporal order memory were reduced or even eliminated by an hour of practice. Having participants repeat the procedure with a second text would have sorted this out.

In presenting the text, Mangen et al. stripped both texts of the page numbers, and covered a progress indicator present on the Kindle. Concealing the page numbers was at least equivalent between conditions. However, concealing the progress indicator seems hard to justify. The progress indicator is a substitute for the physical affordances of books, to give readers exactly the sense of location that Mangen et al. were measuring. Although not all ebook reading software provides the same kind of progress indication, removing it from the e-reading condition but not from the print condition stacks the deck against e-reading. The authors did nothing to remove the physical indicators of position from the print condition (for example, they could have used a ring binding).

What does this imply for real-world e-reading? The most straightforward conclusion would be that if you remove all positional cues, readers have trouble remembering positional information. The authors did so on the Kindle, and they did not do so with the print book. Since this is not a normal way to read ebooks on the Kindle, perhaps it’s of no concern at all. On the other hand, since not all e-reading software provides always-on progress indicators, maybe it does point to a real (but easily surmountable) disadvantage.

There was also a small numerical difference between the two groups in self-reported reading frequency – subjects in the print condition reported reading slightly more. Perhaps the groups were sufficiently closely matched and perhaps not. It’s conceivable that the difference accounts for some or all of the results. In any case, I don’t believe it was factored out in any of the analyses. On the other hand, the average self-reported reading frequency of participants in the study was about halfway between 3-5 books/year and 5-10 books/year. Given that college students are expected to read for their coursework, these are shockingly low numbers, suggesting few avid readers in the group. Perhaps Mangen et al. would have observed a greater advantage for print had they examined more dedicated readers.

I will also take issue with the choice of the Kindle DX, a 10-year-old design last manufactured in 2012, that presents text at 150ppi. Modern reading devices typically have a pixel density closer to 300ppi (what you’d get from a cheap laser printer), and the DX was criticized for this even when it was new. Newer devices also have smoother page turns and better contrast. Although it’s a reasonable choice given this study’s aims, it clearly offers a poorer e-reading experience than modern devices.

Finally, the authors report non-significant differences on most measures of comprehension. Testing a large number of measures and finding one reliable one is often a sign of statistical artifact. In this case, they found three differences, although those three measures are unlikely to be independent. The other measures show a mix of numerical advantages both for print and for e-reading. However, the authors report a specific prior interest in temporal processing, which gives them some leeway to report this finding without the normally required statistical correction.

Overall, I think this study presents a more solid basis for investigating purported differences between e-reading and print than most of the literature. There’s some genuine evidence that people unfamiliar with e-reading may have trouble placing textual events in temporal order when all positional cues are removed, at least on the venerable Kindle DX. This is an interesting finding worth pursuing, but I don’t think it’s a smoking gun for an e-reading disadvantage.

Why aren’t there more decisive studies?

It’s not anyone’s responsibility to answer the specific questions I have. There are very good reasons why someone would have a research interest on the consequences of reading things in a web browser, PDF reader, or in Microsoft Word on a desktop computer. Or on an early model Kindle, older LCD panel, or even on a CRT screen. Studies using these kinds of materials aren’t relevant to the questions I personally care about, but that’s not a fault of the studies.

That said, I have a few broad criticisms:

Reliance on dated technology. The literature will always lag a bit, because technology keeps improving, and academic researchers don’t always have the funding they need. But it surely lags more than it needs to. This will eventually work itself out, as the rate of improvement in the technology levels off. But in the meantime, it’s not easy to draw conclusions that are relevant in 2020 from studies using devices from the dawn of e-reading.

Lack of attention to prior ebook reading experience. Giving participants significant experience during a study can sometimes be impractical. However, I was surprised to see how little attention is given to the fact that college undergrads, however fluent they may be with digital media, typically have no experience with reading ebooks. Few studies reported even measuring e-reading experience, let alone trying to factor it out of analyses.

Handicapping ebooks. Many of the advantages of ebooks were unavailable to readers in these studies. Font choices (typeface and size), illumination, the size, weight, and screen type of the device, and various other factors were generally selected by researchers. While this may make interpreting the results superficially more straightforward, it also makes the results much less relevant to the question of how you should read. Because if you were to choose e-reading, you would not be subject to these same constraints.

Inadequate reporting of methods. I have to keep harping on this, because it’s important. “A standard 20 inch computer screen” tells us almost nothing about how an ebook was presented. Was it a CRT, a high-end OLED, or something in between? Were materials presented at native resolution, and what was that resolution? What typefaces and font sizes were used? Was the text hand-held (as in a tablet) or affixed to a desk (as in a monitor)? How much did the materials weigh? What was the page turning mechanism? And so on. Scientific reporting always requires some judgment in deciding how much detail to include. But these studies are about screens. Although some studies do a better job than others, I was surprised by how often I found the methods reporting deeply insufficient, and by how often I found language suggesting that the authors were, to put it mildly, not very familiar with technology.

Focus on “desktop” reading. Reading on a desktop or laptop is a great convenience, especially if you’re looking to retrieve a specific bit of information or if you’re just skimming. It’s not, in my experience, how people who care about getting the most out of their reading consume long form content in their dedicated reading time. In some cases, it appears that studies may have compared fixed computer monitors with more portable printed materials.

Willingness to over-interpret the data. Notwithstanding some post-hoc hand-waving, few of the studies I’ve seen have any real theory about what underlies the differences they report. But the willingness of authors to conclude from narrow and sometimes bizarre reading conditions that e-reading is generally bad for you struck me as beyond the normal scale of academic overinterpretation. While I expect this from news reports, it’s evident to some extent in the journal articles themselves.

I’d like to expand a bit on this last point. Articles in scientific journals often have a lot of non-scientific content. In my old field, journal articles often reserved a section at the end for “discussion,” an author’s high-level recap of their findings. This is where authors are customarily allowed to do a bit of hand-waving. And it’s arguably okay, because everyone understands the custom. I don’t work in the field anymore, but I still know how to ignore those bits. Writers in the popular media, unfortunately, are less likely to make this distinction, and probably more likely to mistake the study author’s preferred interpretation as a finding of the study.

I can’t help feeling let down by some of the academics who’ve contributed to the feeding frenzy of popular media condemning e-reading. Although I certainly understand the distortion that can come when academic research is passed through the filter of popular media, I wish the original academic reports were more forthright about their limitations. In Richard Feynman’s well-known “Cargo Cult” speech, he takes social scientists to task for lacking the scientific integrity to report honestly and comprehensively on all of the shortcomings of their theories. For failing to bend over backwards to examine all of the ways in which their conclusions might be wrong. This was something I felt acutely during my many years in academia, and I suffered for it (at one point, my department chair told me to stop declining authorship on articles I felt were substandard).

It’s easy to see how, if your personal biases favor print reading, you would find some support in the literature. But it’s the duty of a scientist – arguably the most important duty – to look at data skeptically. To consider the evidence and to say to oneself, “If I started with completely different beliefs, would the data before me convince me that I was wrong?”

In the case of e-reading, I don’t believe that evidence exists.

The importance of going to the source

The scientific literature on e-reading often bleeds into more popular media in the form of short news articles. Over the past decade, I’ve seen dozens of such articles in venues as widely read as The New York Times and Wall Street Journal, and many smaller. I’m sure I’ve missed hundreds. But as any scientist will tell you, popular media is not a reliable source for science reporting. Real scientific findings are distorted and simplified in order to provide a digestible story or the most catchy headline, even if that headline, or the story itself, is completely misleading. As a scientist, I saw this countless times in reading articles about research I know well.

So when I see articles about the drawbacks of e-reading, I get concerned. But not deeply concerned, until I get to the bottom of it. And now that I’ve gotten to the bottom of it, I don’t think there’s any solid data to put me off ebooks. The studies I’ve seen are completely consistent with my belief that e-reading can be just as effective as print reading, and likely more so. But you never know what the next batch of studies will show.

Why does this all matter?

Because students are the largest reading demographic, and students don’t always make their own technology decisions, how we read is partly a matter of public policy. If we want to make good public policy decisions, we can’t base those decisions on research drawn from what happens in your first hour, or even day or two, of e-reading, and we can’t base those decisions on studies with unrealistically poor reading conditions. E.g., if reading Shakespeare on a Kindle turns out to be just as effective as reading in print then we would be foolish to abandon the advantages of e-reading in schools just because reading a PDF on a CRT or on a cell phone is a bad idea. On the other hand, if some students form a strong dislike for e-reading, and that affects their ability to learn from this medium, it would be important not to force them to read ebooks.

It also matters for the rest of us. E-reading technology is still in its infancy, but it already offers many advantages over print reading (see my previous article for a run-down of both the advantages and disadvantages). Everyone has to balance the considerations in their own way. But it would be harmful to convince someone who might otherwise benefit from e-reading to stick with print for reasons that don’t apply to them.

My Bottom Line (For Now)

It’s quite clear that reading on screens can be worse than reading in print. But what jumps out from these studies is not that e-reading is inherently worse, but that it’s possible to do it poorly. It shouldn’t be surprising to see poor performance if you force people with zero e-reading experience and a strong affinity for print to read documents in Microsoft Word on a CRT. Anyone who’s conversant in technology should immediately recognize that as a bad idea.

It’s also possible to do dumb things in print. We could use ornate typefaces in tiny font sizes, printed in smudgy red ink on broad sheets of green paper, in wide columns and with dim lighting. But everyone knows not to do those things. E-reading is comparatively new, and that inexperience is reflected in much of the scientific literature. If you’re going to read ebooks, you can do much better than the typical conditions in these studies.

For me, the bottom line right now is that although e-reading is still in its infancy, there’s no reason to believe it’s inferior in any way to print. There will be more bottom lines as more research comes out. I’ll be sure to post an update if I see anything really decisive.

Appendix: What did I actually read?

I started on the literature by reading Naomi Baron’s 2015 book, Words Onscreen, a now slightly dated book that is both a peek at the scientific literature and an opinion piece. I wanted to start with an overview of the argument against e-reading, and Baron is a card-carrying academic.

From there, I proceeded to two meta-analyses (Clinton et al., 2019 and Delgado et al., 2018; meta-analyses are studies that combine data from earlier studies) and a literature review (Singer & Anderson, 2017), in order to get a handle on what’s been done.

Finally, I both read and skimmed a large number of individual studies (I don’t cite them individually here, because this is a blog, and it would make a mess), including many of those cited by Clinton, and a handful published since. Studies prior to the first Kindle are largely irrelevant to modern e-reading, and as a practical matter, I draw the line closer to 2012, when both Kindles and tablets were starting to mature. But since Clinton’s 2019 meta-analysis (which turned up a mere 29 usable articles) went back to 2008, I looked at some of those as well.

There are some things I missed. Although most of the literature is freely available for download, some are protected by paywalls, with costs ranging from $24 to over $100. If you work at a major university, your library maintains a subscription to these services, and you don’t have to worry about it (well, you should, but that’s a different story). I don’t, and I refuse to pay for access to this kind of work. In two cases, I’ve written directly to study authors for “reprints” (dated academic jargon for “please send me a copy”). One was very kind, the other never replied. So there’s at least one article out there that shows an advantage for print, and that I thought was worth reading, but that I couldn’t get my hands on.

Appendix: My Academic Credentials

I don’t like getting hung up on credentials. But since I’m asking you to trust that I’ve done a decent job distilling the literature, I should say something. I got my PhD in cognitive psychology from Carnegie Mellon in 1994, and spent the next 17 years in various academic positions. My final academic job was a “research faculty” position – I never pursued a tenure-track faculty job (probably a bad career move) or held a teaching job. Most of my time in academia was spent in cognitive neuroscience, a multidisciplinary field that draws on cognitive psychology and neuroscience. I don’t have a background in the psychology of reading, and although I’ve collaborated with language researchers, it’s not an area of specific expertise for me.

In 2011 I got an offer I couldn’t refuse, and left psychology for software engineering.

Appendix: Reference notes

This isn’t a scientific article, and I’ve mostly avoided naming specific studies above. But if you want an entry into the literature, here are the starting points I’ve referenced above.

• Naomi Baron’s 2015 book “Words Onscreen” is a great overview of the arguments against e-reading.
• Clinton’s meta-analysis.
• More recent studies citing Clinton.
• The 2019 study by Mangen et al.

Disclaimer

Although I do work for an ebook seller (Google Play Books), this article reflects my views and not those of my employer. I’ve been a devotee and advocate of e-reading since well before I started working in the tech industry, and long before I started working on ebooks. And I do still use three different e-reading products, though mostly the one I work on.