Keywords: benefits, impact assessment, clinical tools

Citation:
Jones, P. (2001) Information Communication Technologies and Benefits Realisation,
<>, Accessed

Introduction

Sadly (as a nurse and tax payer), public sector computing - including health - has been littered with unsuccessful projects, with descriptions ranging from 'problematic' to 'disastrous', according to the UK press: The Guardian, Computing, Computer Weekly. The realization of benefits must be taken seriously. From the outset ICT enthusiasts should beware of making claims that create disillusionment and frustration in the would-be user group.

As with the wider economic and commercial world information technology should improve productivity within health care. Providing benefits via modern information management techniques via formalization of many aspects of data gathering and updating using computerised care plans, care templates, collaborative care plans and electronic patient records. As we enter the 21st Century continual advances are needed, in the UK and worldwide a shortage of nursing and other personnel increases the burden on existing staff with effects that can include burnout and premature loss of many experienced staff.

Everyone would agree that whilst technology is a tool, the problem of realizing benefits is not helped by the increasingly complex working and personal lives people live. It is no longer just a matter of replacing a manual filing system - which can prove problem enough - but the impact of ICT is now intra and inter-organizational and local, national and international.

The implementation of ICT is analogous to climbing a mountain. Even before arriving at base camp a management plan is formulated, and agreed based on needs and goals. A procurement process is enacted, one that is balanced; not stifling, smothering in its bureaucracy, but disciplined, flexible, responsive to market conditions and 'corporate' needs. Procurement that keeps an eye on the legal route to follow, overall goals, the realities of commercial life (strife?). The ascent of the mountain (by this multi-eyed creature) must be project managed every step of the way. Once at the summit, enjoy the vista, then on the descent reap the benefits. Benefits that you planned for and hopefully a few you did not. Once back at base camp your life cycle management plan 'kicks-in' - becomes the focus - as the next climb beckons - future strategy. Mountains aside - what are benefits anyway?

What are benefits?

One thing benefits should not be - is left to chance. Benefits should also not be limited to ICT, but what are often termed 'health technologies'. One informal definition and explanation of what health technologies are is provided by the following:

"all the methods used by health professionals to promote health, to prevent and treat disease, and to improve rehabilitation and long term care." DoH (1992)

While the formula below provides a form of definition, isolating the factors that influence the accrual (or otherwise) of benefits is a struggle of the most dynamic kind. A great deal of research exists on benefits realization in health informatics and computing generally. Rosenstein (1999 a,b) details quantification of the return on investment in clinical decision-support systems. Szczepura & Kankaanpaa (1997) provide a formula in which:

V is the present value of technology. The time-frame of analysis extends from t=0 (the present) to some future = T. The Greek capital sigma means summation across all time periods t. B is the level of benefit (or outcome for each time period t. C is the level of costs for each time period t and r is the discount rate. Szczepura and Kankaanpää (Eds.) (1997) p.51

Recognizing benefits obviously demands comparison. How does 'x' - the way we work now - compare with the former way 'y'? It is important that the same things are compared (or the same questions asked) to ensure accurate conclusions are reached. Objective measures are needed. Using an example from human-computer interaction Whiteside et al (1987) list possible measurement criteria at one level:

1. Time to complete task.
2. Percentage of task completed.
3. Percentage of task completed per unit time (speed metric).
4. Ratio of success to failures.
5. Time spent in errors.
6. Percentage number of errors.
7. Percentage or number of competitors that do this better than current product.
8. Number of commands used.
9. Frequency of help or documentation use.
10. Time spent using help or documentation.
11. Percentage of favourable/unfavourable user comments.
12. Number of repetitions of failed commands.
13. Number of runs of successes and of failures.
14. Number of times the interface misleads the user.
15. Number of good and bad features recalled by users
16. Number of available commands not invoked.
17. Number of regressive behaviours.
18. Number of users preferring your system.
19. Number of times a user is disrupted from a work task.
20. Number of times the user loses control of the system.
21. Number of times the user expresses frustration or satisfaction.

Amongst all the values and formulae, the most vital variable is still left out. People: the USERS. Do not be mistaken the users are a variable - they vary in:

skills,

attitude,

and motivation,

when it comes to most things - especially ICT. It must be remembered that infrastructure starts with people and not just clinicians. Secretarial and administrative staff also have crucial roles to play in the success of ICT projects.

People: More than models, more than beans

Amongst all the models, theories and formulae, the most vital consideration is still often left out. People as social beings, with a gamut of priorities and stimuli demanding attention. People as users of information and information systems. What nurse and health theorists need to discern are intrinsic features of the HCM, what Gibson (1977) called affordances. These are purposes or application that are extensions of the original purpose of an artifact. An example being the use of fridge magnets on the door, for decoration and posting notes. Nardi and O'day (1999) stress how researchers now:

'understand that many tool affordances have an important social dimension.'

It is only in use that we become aware of affordances associated with a tool, perhaps this accounts for the way that ICT projects do not achieve their objective. Affordances are obvious in some cases such as the example of the pencil and its tool affordance mentioned by Nardi and O'Day. Yes, it fits in the hand, behind the ear, often has an eraser built in, and of course make an erasable mark; but what are HCM's affordances? It is these we need to anticipate, verify and measure if at all possible, but the HCM and other (candidate) conceptual tools are not pencils. And by their nature affordances are subtle and only become apparent post-implementation.

Health care is (supposed to be) very fact based. Yet as a science based discipline, hypotheses must have their day whether 24 hours long or just fleeting seconds. The HCM could certainly and should most definitely be used to record, analyze and manipulate factual information. Perhaps, there is an affordance however, in its use to generate hypotheses about the patient and their wider context. McGoldrick, Gershon and Shellenberger (1999) in their text on genograms (family trees), provide a caveat (and an opportunity) :

Throughout this book, we make assertions about families based on their genograms. These observations are offered as tentative hypotheses, as is true for genogram interpretations in general. They offer suggestions for further exploration. Predictions based on the genogram are not facts. The principles for interpreting genograms should be seen as a road map that, by highlighting certain characteristics of the terrain, guides us through the complex territory of family life - nothing more. p.11.

In application the health career model may facilitate the generation of hypotheses, NOT JUST through the complex territory of family life, but the territory through which families must pass.

No account of benefits can fail to mention the scepticism that clinicians ascribe to ICT in their work. Many promises have been made, and we all know what you should not do with promises. "If you can't keep them - don't make them!" If governments and civil servants want health services that can account for themselves, where management data [information] truly is a by-product of care delivery, then the 'bean' counting must stop, or at least be accompanied by qualitative measures.

Personal experience and policy review highlights a shift. Politicians recognize the expediency of giving the public what it wants. Change is afoot. Lessons are being learned, at some levels anyway? As tax-payers, nobody likes to see public money wasted. Critics of public/governmental computing (the computing and general press) must realize that if they "leave the knife in", the wounds will have trouble healing.

Clinically, Martin, Taylor & Kearns (1997) point towards the start of trends in long-term care in the USA:

changing regulation of the nursing home industry with emphasis on outcome and assessment, a trend in medical informatics away from expert systems and toward on-line decision support and reminder systems, and the application of industrial statistical quality management techniques to the realm of human services. Emerging standards such as the Arden Syntax and Unified Medical Language Systems and technologies such as Rapid Application Development Tools will facilitate the use of modern computing to mold and implement these converging trends.

In the UK the DoH work progresses on a Mental Health Information Strategy. The consultation process has engendered similar views, seeking a more person focused, outcome centered approach.

Usability

Usability in this context is the design of a system that provides an efficient, effective way to get a task done.

It is all too easy to get carried away with the interface. When "how does it look?", becomes more important than "what can it do?" It is still surprising how many people - those with little ICT experience - are seduced by a 'well presented' interface. This is one way our ocularcentric culture reveals itself, but (ironically) the costs of this tendency remain largely hidden. Experienced ICT users are aware of the ease with which an interface and bare-bones application can be "knocked together" After all that is what rapid application development is (was) all about? Naive users, however, may not try to 'chip the paint', to understand the depth, mechanics and conceptual basis of what they are being shown.

Usability has its critics as elucidated by Booth (1989), but the principle could still teach us much about applying ICT in health care. For example, how would the HCM fare from a 'usability' point of view? The HCM quadrants lend themselves to assessment, but what about care planning and recording interventions and outcomes?

There are, so it seems, two (contradictory?) cravings within health and social care. On the one hand - each discipline seeks to maintain its status, role and future. Simultaneously, policy, managers, patients and professionals themselves seek to blur their roles and work in multidisciplinary harmony. Whether these objectives are mutually exclusive is a moot point, but Brian's work might prove an invaluable tool in reconciling these two needs and achieving usable systems.

Initially, there were formal and informal interfaces between professionals, the Consultant's ward round, case presentation meetings. Such activities centered on the institution. Next the shift to community oriented services began, with (the intended) integration of 'teams' often including consultants. Today, we have (supposedly) arrived at interactive teams were those professionals engage in high level interaction (joint therapeutic intervention) not just with each other, but with clients and their carers, the latter often being assessed in their own right, and the very focus of intervention. Ovretveit (1998) clearly shows that there are several models for integrated care and an equal number of issues that arise.

What is the outcome of this? Obviously increased complexity and greater need for connectivity; integration of internal and external resources (multi-organization, e.g. Social Services, Voluntary Sector, Health); then what must happen is communication, without which interfaces exist but are silent, integration amounts to words on paper (sharing office space), but interaction is haphazard, stilted, unrewarding, or even avoided.

Integrated Care Pathways have been proposed as a solution to ensuring efficiency and quality outcomes in coordinated multidisciplinary care. A potential that may be enhanced via use of communications and web-based technologies. Surely though what is needed are Interactive Care Pathways? Is integration an automatic outcome of interaction? What exactly is the right mix of ingredients? Or is it wrong to expect formulaic solutions?

Interfaces get in the way. I don't want to focus my energies on an interface. I want to focus on the job. Don Norman

The clickable figure above shows the Usability Iceberg. The parts of a project that often have the most influence on people - the interface - is the part that contributes least to the systems usability. Emphasis and effort must be placed on the conceptual model, and the wider social context of a systems use.

Why does the conceptual model require 60% of the iceberg in the above figure? Well the explanation is implied in the image, the conceptual model forms the foundation for, underpins the way the user(s) interacts with the system and what is presented and how. A criticism of early computer aided learning software was that it amounted to little more than 'electronic page-turners'. Nielsen's Taxonomy of System Acceptability also sheds light on the hidden 60% that can wreck even the best project management pilots. Contemplate Nielsen's taxonomy, and it becomes clear why the electronic medical record, electronic patient record, or electronic health record must be more than a clinical file/case note.

Sujansky warns us that:

The potential of such tools will not be realized, however, if the EMR is just a set of textual documents stored in a computer, i.e. a "word-processed" patient chart. To support intelligent and useful tools, the EMR must have a systematic internal model of the information it contains and must support the efficient capture of clinical information in a manner consistent with this model. Although commercially available EMR systems that have such features are appearing, the builders and the buyers of EMR systems must continue to focus on the proper design of these systems if the benefits of computerization are to be fully realized. Sujansky (1998)

The iceberg can also be used to illustrate two difficulties that seem to confound or sink any aspirations of ICT realized benefits. Information retrieval is a constant activity at several levels, between people and machines. Spence and Tweedie (1998) discuss several specific shortcomings of traditional information retrieval methods:

'Thus, six drawbacks associated with conventional database query interfaces have to be addressed:

• The discretionary user must learn a language.
  
• Errors are not tolerated.
  
• Too few or too many hits may be registered.
  
• No guidance is given regarding potentially beneficial modification to the query.
  
• Useful contextual data are often unnecessarily hidden.
  
• It is difficult for a user to build an internal model.'

The notion of usability can be extended to health services. For example, patients may understand their care plan, if it is in terms they can understand. New interfaces may use graphical metaphors and analogies to explain, plans, events, and outcomes, but usable systems must extend beyond 'graphical gimmicks'. Otherwise the graphical front end will lead to a back-end dependent upon text.

How often are users tempted and teased with graphical interfaces, only to meet pages of text, after that first double click of the mouse or 'pen'? What should the balance be? Text has its uses! Are there any metrics that can be applied? For governments and patients alike it looks 'modern' to have health care staff using the latest PDAs, but can they use them?

Clinicians can inform ICT practices

Clinical experience and research (in psychoses) can inform ICT practice. Two researchers Zubin & Spring (1977) suggested that:

"As long as the stress induced by challenging events stays below the threshold of vulnerability, the individual ... remains well within the limits of normality. When the stress exceeds the threshold, the person is likely to develop a psychopathological episode of some sort [a new illness or a relapse of a previous illness ] ... when the stress abates and sinks below the vulnerability threshold, the episode ends."

THE MORE VULNERABLE THE PERSON IS

the LESS stress

IS REQUIRED TO PRECIPITATE ILLNESS.

It is possible to adapt Zubin and Spring's model to ICT. In the illustration to the left (please don't click just yet), several users are placed along the base, each person has their own unique vulnerability to information overload. Some individuals such as hybrid managers possess ICT skills and management acumen. They are info-adapted. As a result they will be found to the far left in the diagram. Given their ICT skills they have a higher threshold before stress affects them. Excessive stress pushes the individual into the territory above the diagonal white line - vulnerability threshold. If exposed for prolonged periods, people are in danger of information overload, or indeed information starvation (uncertainty) - such is the perverse nature of chaos.

There are many crucial factors not included in this adaptation of the stress-vulnerability model: such as, finance, and life cycle of systems. What the diagram (hopefully) illustrates is the basic interaction of key variables. Variables that are often hard to articulate and utilize to inform the current and future ICT projects of whatever scale and complexity. Graphical representation could help develop hybrid approaches that facilitate crossover between healthcare/nursing theory and practice, (resource) management and informatics.

We must be able to see a tool, check its functional potential (like the proverbial swiss army knife), before we can pick it up with confidence, use it and derive any benefits whatsoever.

© Peter Jones 2000

References

Booth, P. (1989) An Introduction to Human-Computer Interaction, Hove, LEA.

British Library (1994) R&D Report 616 Information Systems for Nursing Specialists, Bawden D Robinson K, Boston Spa

Department of Health (1992) NHS Standing Group on Health technology, Assessing the Effects of health Technologies: Principles, Practice, Proposals. London, DoH.

Gibson, J.J. (1977) The theory of affordances. In R. E. Shaw & J. Bransford (Eds.), Perceiving, Acting, and Knowing. Hillsdale, NJ: Lawrence Erlbaum Associates.

Martin, D.C., Taylor, D.P., Kearns, L. (1997) An information infrastructure for long-term care. Source Topics in Health Information Management, 18: 1, 10-21.

McGoldrick, M., Gershon, R., Shellenberger, S. (1999) Genograms: Assessment and Intervention, 2nd ed., WM Norton & Co., London.

Nardi, B.A., O'Day, V.L. (1999) Information Ecologies: Using technology with heart, MIT Press, London.

Norman, D.A. (1990) The design of everyday things. New York: Doubleday.

Ovretveit, J. (1998) Integrated care – development issues from an international perspective: integrated care: models and issues. Healthcare Review – OnlineTM. 2(5); March 1998. http://www.enigma.co.nz/hcro_articles/9803/vol2no5_001.htm

Rosenstein, A.H. (1999a) Measuring the benefits of clinical decision support: return on investment. Health Care Management Review, 24: 2, 32-43.

Rosenstein, A.H. (1999b) Inpatient clinical decision-support systems: determining the ROI. Healthcare Financial Management, 53: 2,51-5.

Sujansky, W.V. (1998) The benefits and challenges of an electronic medical record: much more than a "word-processed" patient chart, Western J. of Med. 169: 3, 176-83

Szczepura, A., Kankaanpää, J. (Eds.) (1997) Assessment of Healthcare Technologies, Wiley, Chichester, 51.

Tufte, E.R. (1983) The Visual Display of Quantitative Information, Graphics Press, Conn., 13.

Wallum, R. (1995) Using Care Plans to Replace the Handover, Nursing Standard, 9,32,24-6.

Whiteside, J., Bennet, J., Holtzblatt,. (1987) Usability engineering: our experience and evolution. In Handbook of Human-Computer Interaction, Helander M (ed.), North-Holland, Amsterdam.

Zubin, J., Spring, B. (1977) Vulnerability: A New View on Schizophrenia Journal of Abnormal Psychology 86, 103-126.

Chiann-Ru Song (2003) An analysis of branching behaviour patterns in an interactive hypermedia learning environment, Online Information Review, 27: 3, 196-206.