Is the recent media buzz over findings that some docs seem to order too many tests electronically a reason to abandon eHealth? Not really. Here’s why…
Eons ago (I’m speaking in technological time), someone thought it was a good idea to adopt information communication technologies (ICTs) to health care, and they called that project “eHealth”. Back then eHealth was probably a good name, too: the health care system was primarily paper based, while most (okay all) other sectors, and society at large, were rapidly adopting electronic information systems as part of their daily activities. Putting the “e” in health was a way of explaining what the big idea was to the average person who, at the time, was still wrapping his head around email (another industry that has since been “e-ified”). At the same time, arguing that we should move towards eHealth was a way of suggesting that the various players in health care—nurses, doctors, hospital administrators, average Canadians, and so on—could benefit from ICTs.
Eons went by (technological time once again), and the Internet changed the world, for better and worse. On the balance, I tend to argue that the benefits of these changes outweigh the bad stuff. Email, Google, Wikipedia, iTunes and, yes, even Facebook, are just a few of the refined technologies that appear to have come out on the positive side of the balance.
Health care, on the other hand, is still in its Internet infancy. Most health information systems are not fully digital, most of those that are do not talk to one another, most doctors do not use electronic health records, and a critical group of players—ordinary citizens—have little or no access to digitized versions of their health data.
It’s bizarre and ironic, given all the beneficial stuff that’s happened over there on the Internet, that some people are still questioning whether or not eHealth, as a broad initiative, is worth investing in. But it seems that is, indeed, the going concern of some prominent health commentators.
Most recently (and quite ironically) the media went into a state of frenzy over an article published in Health Affairs, which appeared at first glance to suggest that doctors with access to electronic test results ordered more tests. Upon reading the report (more likely, I suspect, the press release by Health Affairs), media outlets like the Washington Post, the New York Times and the CBC ran stories (which found their legs on social media sites to a large extent) suggesting that the study legitimately called into question the entire eHealth agenda.
In response, Julia Belluz posted an article on the MacLean’s blog “Science-ish”, that was, in part, a call for more research into the benefits of eHealth.
Fair enough. Not much research has been done on eHealth technologies, and it’s always nice to have evidence to back up your claims. This lack of evidence might seem shocking considering the nearly $4 billion that has been spent on eHealth in Canada to date. I take no issue with a call for more research into the various technologies that comprise the larger system we still refer to as eHealth.
However, Belluz didn’t merely call for more research into eHealth. In addition, and in contrast to her colleagues, she pointed to the many obvious problems with the paper that had caused so much of a stir in the first place. Quite rightly, she dismissed its findings almost entirely.
Why? The paper says little about the technology it studied (there were serious problems with the study), and next to nothing at all about “eHealth” in general.
In addition to the problems Belluz points to, I’ll add the following caveat for anyone reporting on a study related to a particular “eHealth technology”: The fact that software X results in negative consequence Y, is not evidence that it does not have any benefits, nor is it evidence that can generalize to eHealth more broadly. It is evidence that software X had negative consequence Y. It’s evidence of nothing more, nothing less.
Unfortunately, the journalists and editors at the Washington Post, The New York Times, and the CBC (among others), seem to have overlooked this important relationship between any particular eHealth technology, and the broader project of eHealth.
An analogy might be useful. The fact that there is evidence that Facebook raises considerable privacy concerns (and even violates individuals’ privacy in many cases) is not evidence that there are no benefits associated with using Facebook (though there may not be). The fact that Facebook raises considerable privacy concerns is also not (most emphatically not) evidence that the Internet is of no benefit to society.
But that is precisely the argument those journalists made against eHealth in their articles. They argue that because there are a couple of pieces of software out there that seem to offer little or no benefit, or that seem at first glance to have the opposite of an intended effect to the health system, that we should be wary of the entire eHealth project.
The problem here seems to be in understanding what eHealth consists of. It is not a single piece of software, any more than Facebook is the Internet. eHealth is the broad and multi-faceted application of ICTs to the health care system. It is much more than any one (or thousand) of its components.
Furthermore, journalists and policy makers need to be measured in the demands they make for evidence. For example, demanding evidence of “eHealth’s benefits”, as if you can provide such a thing, is very much like demanding evidence of the benefits of the Internet. It is a fool’s game if ever there was one.
What we can do is look at particular technologies, used in particular places, by particular people, and try to gauge if they are delivering certain benefits, or not, in that context. If they are, great! We can then try to leverage those benefits in other contexts.
What happens if a technology looks like it isn’t delivering on a particular intended benefits, as seems to be the case in the study so widely quoted? Should we call for the abandonment of eHealth?
That would be hasty. In fact, it would be hasty even to call for the abandonment of that particular technology without first balancing the known problems against the known benefits (something the paper did not do). It might turn out that that technology is not beneficial overall. So be it. Most technologies end up in the dustbin—but that seems just to be a consequence of pursuing new technologies. It might also turn out that the benefits outweigh the costs, in which case the appropriate response would be to research and tweak the program to try to improve whatever is causing the unwanted effect (too many tests in this case).
But that is a very different type of research than “studying eHealth”.
The fact is, nobody really “studies eHealth”, just like nobody really “studies the Internet”. Rather, they study bits and pieces of them—social networking technologies; routing algorithms; communication standards, privacy and so on—to try to come to some simple, hopefully interesting and helpful, conclusions.
The upshot is that recognizing where “eHealth” research fits into the eHealth story says a lot about how the media ought to communicate scientific research more generally. A journalist might point to a study showing that a particular piece of software that allows American doctors practicing at such and such a hospital to order tests electronically, results in more tests ordered. That journalist can then say something about it being problematic that American doctors in such and such a place ordered too many tests (if that is, in fact, what they did). That journalist cannot say (with a straight face, anyway) that that study suggests “eHealth” fails to produce benefits generally.
Similarly, when I point to this evidence, and this, which suggest that medication errors drop significantly (a statistical claim) when electronic ordering systems are used, I can only conclude particular things. I can conclude (assuming the research was good…and it appears to be, despite the usual worries about bias and generalization) that the particular software systems being studied tended to reduce medication errors in those instances under study. I can hypothesize that if other health care institutions adopt these technologies, then fewer medication errors will be made. If they adopt them in particular Canadian health care settings, I can hope that Canadians will benefit sinilarly, in that they will experience fewer medication errors.
Though the research on drug ordering systems says very little about eHealth insofar as eHealth means the broad application of ICTs to health care, it does suggest that we would be wise to look into electronic drug ordering systems for our hospitals. Which is exactly the kind of thing that I suspect hospital administrators, the folks at Canada Health Infoway, Provincial and Territorial governments, and others who want to realize some of the obvious benefits they’ve experienced over there on the Intenet, are doing with some of the $4 billion dollars we’ve collectively invested in eHealth.
Journalists need to move beyond claims that “eHealth is beneficial,” or that “it isn’t”. They’ve certainly moved beyond claims that “the Internet” is such and such. Evidence supporting such broad claims isn’t likely to appear, however much it would simplify the storytelling. Journalists need to give up the eHealth moniker, and instead talk about the many eHealth technologies for what they are, specific technologies, bits and pieces of a very complicated and ever changing puzzle. Each one of those technologies has the potential to confer benefits, or not. Sometimes we’ll get the technology right and realize those benefits, sometimes we won’t.
Characterizing “eHealth” according to the qualities of a few transitional technologies does very little, if anything, to inform the public about their health care or the system that delivers it. Nor does it help the public decide whether or not eHealth is a worthwhile endeavour.
Jason Millar is a PhD Candidate at Queen’s University, and currently holds a CIHR Science Policy Fellowship in eHealth. He studies in his corner of the eHealth world, trying to figure out how various consumer health technologies ought to be used to engage and benefit patients.