Evidence Based Health IT – What a Good Idea!

In what was almost a throw away line commenting on the recent discussion regarding archetypes Dr Tim Churches of NSW Health offered the following remark.

"My view is that the practice of health informatics needs, desperately, to become evidence-based, otherwise we will continue to see hundreds of millions or billions of dollars being poured into the deployment of health information systems based on what is in the sales brochure, or based on tender responses, which tend to be just more elaborate versions of the sales brochures."

Now, while I may disagree that the only current driver of Health IT deployments are sales brochures I find the broad sentiment most compelling. Tim is right in suggesting that the decisions regarding adoption and use of Health IT should be on much the same basis as we seek to consider and manage other Health interventions (drugs, treatment protocols, medical devices and so on).

This led me to consider what such Evidence Based Health Care IT may look like. What follows are my initial thoughts on how the idea could be actualised.

First it would be necessary to decide on the type of evidence that was being sought and in what domain the decisions would be assessed. The health sector has many years experience in the design and execution of clinical trials and there does not seem to me to be any intrinsic reason why such an approach would not be useful.

One can easily envisage trials comparing manual methods of practice with computer supported approaches as well as trials comparing computer systems with varying levels of sophistication in terms of user interfaces, functionality and decision support capability.

An issue which would need to be carefully considered is how, in a trial situation, the quality of the system is distinguished from the skill of the user and possibly patient related factors. We need to recognise that Health IT is much more an enabler of service delivery rather than a provider of services per se and this fact would need to be reflected in the trial design.

Second it would seem reasonable to use the typical types of trial endpoints for evaluation of clinically relevant systems – perhaps modified and controlled in different ways as appropriate. The use of endpoints, such as clinical outcomes, also has the advantage of taking a holistic view of an intervention and not seeing a Health IT intervention purely as a technical exercise – but also as something that involves the patient and the clinician as well as the technology.

Of course it is possible that in some situations there will be the need to consider intermediate outcomes rather than direct impact on morbidity or mortality when considering, for example, differing decision support systems aimed at improving specific clinician behaviours.

Third there is obviously a place for technical trials of different technical approaches to clinical problems evaluated on both technical as well as clinical criteria. In the same vein one would also want, in some circumstances, to ensure parameters such as cost, impact on workflow or efficiency and so on were evaluated as a component of an overall assessment.

Fourth as experience grows one would see, hopefully the emergence of multi centre trials of particular generic interventions and development of appropriate strategies for the conduct of systematic reviews of important interventions. Later expertise would also be encouraged to develop in cost benefit assessments of the different interventions using criteria rather more closely modelled on the Pharmaceutical Benefits Review Committee than what is done by the Therapeutics Good Administration where the focus is more on safety rather then efficacy and clinical outcomes.

Fifth it is important to consider how best the human side of Health IT deployment and use can be researched and practical trials conducted to come up with the best ways to manage system selection, clinician resistance to change, workflow alterations and so on.

Sixth one could consider the establishment of a Cochrane Collaboration like clearing house to make widely available the outcomes of both sponsored research, systematic reviews and so on – as well as Health IT guidelines to assist organisations move forward with an evidence based Health IT Agenda

Last it might make some sense to have at least some research that looks back on failures and disasters (something much beloved of IT project managers) to come up with evidence based guidelines on what tends not to work in the process of selection and deployment of apparently well considered Health IT interventions and systems.

I recognise that it could be said that what is being suggested is little more than what is classically referred to as Health Technology Assessment (HTA). I would suggest the focus on the deployments of, and the evaluation and trial of Health IT against clinical objectives using an experimental model moves a good deal beyond traditional HTA.

Overall, I can see there could be a case built for establishing an academic centre for the development, evaluation and implementation of Evidence Based Health IT.

What do others think? A good idea?

David.