Article Information

Hendrik J. Loubser1
Judith C. Bruce2
Daleen Casteleijn3

1South African Database for Functional Medicine, South Africa

2School of Therapeutic Sciences, University of the Witwatersrand, South Africa

3Department of Occupational Therapy, University of the Witwatersrand, South Africa

Correspondence to:
Hendrik Loubser

Postal address:
PO Box 2356, Houghton 2041, South Africa

Received: 21 Sept. 2012
Accepted: 18 July 2013
Published: 17 Sept. 2013

How to cite this article: Loubser, H.J., Bruce, J.C. & Casteleijn, D., 2013, ‘The DELTA® nursing measure: Its development and testing for nursing utility’, Health SA Gesondheid 18(1), Art. #699, 9 pages.

Copyright Notice:
© 2013. The Authors. Licensee: AOSIS OpenJournals.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The DELTA® nursing measure: Its development and testing for nursing utility
In This Original Research...
Open Access
   • Problem statement
      • Aims and objectives of study
      • Definition of key concepts
Research design and method
   • Research design
   • The sample
   • Data collection and analysis
Ethical considerations
   • First study: Design and development
   • Second Study: Nursing utility
      • Question 1: Is the DELTA nursing measure useful to the nurses?
      • Question 2: Can it be embedded into the nursing process?
      • Question 3: Can the items of the DELTA nursing measure be used as a uniform language?
      • Question 4: Does the DELTA nursing measure improve the quality of nursing services?
   • Competing interests
   • Authors’ contributions

In the specialised nursing field of acute mental illness nurses expressed a need to measure and evaluate their patients’ mental-health outcomes both empirically and routinely. The aim was to develop and test a measurement tool, named the DELTA nursing measure, which could be embedded routinely into the nursing process and care plans, enabling the psychiatric nurses to score and evaluate their patients’ acute mental-health outcomes. A qualitative, exploratory study design was used to address two sequential objectives. Firstly, qualitative data that described observable behaviours in patients with acute mental illness were collected from psychiatric nurses (n = 5) who were experienced in acute mental healthcare. The data were analysed using inductive content analysis techniques to design and construct the DELTA nursing measure. In the second objective, the nursing utility of the DELTA nursing measure was studied. This was done by training and testing a new team of psychiatric nurses (n = 25) working in a 116-bed acute psychiatric hospital, in the application of the DELTA nursing measure. After 30 months a focus group (n = 6) representing this team was held to explore their perceptions and experiences of the nursing utility of the newly-developed measure. The descriptive data were analysed using deductive content analysis techniques. The outcome of the DELTA nursing measure as a routine nursing measure of acute mental illness provided good results. The nursing-utility characteristics have confirmed positive responses with regard to its acceptance, usefulness and confidence as a worthwhile tool to be used in expediting nursing services in acute mental healthcare. The positive responses to the DELTA nursing measure are noteworthy. It has the potential to add substantial value to the mental health care field in nursing by adding a measurable dimension to patient outcomes, a much needed requirement by patients, multidisciplinary teams and healthcare funders.


In die gespesialiseerde verplegingsveld van akute psigiatrie het verpleegkundiges ’n behoefte uitgespreek om roetineweg pasiënte se geestesgesondheidsuitkomste empiries te meet en te evalueer. Die doel van die studie was om ’n meetinstrument, genoemd die ‘DELTA nursing measure’, te ontwikkel en toets wat ingebed kan word in die verplegingsproses en versorgingsplan en wat psigiatriese verpleegkundiges in staat kan stel om roetine-weg die pasiënte se akute geestesgesondheidsuitkomste te meet en te evalueer. ’n Kwalitatiewe, eksploratiewe studie ontwerp is gebruik om twee opeenvolgende doelwitte aan te spreek. Eerstens is kwalitatiewe data wat observeerbare gedrag in pasiënte met akute geestessiektes beskryf ingesamel van psigiatriese verpleegkundiges (n = 5) wat ervaring het in akute geestesgesondheidsorg. Die data is geanaliseer met induktiewe inhouds analiese tegnieke om die ontwerp en konstruksie van die ‘DELTA nursing measure’ te bewerkstellig. In die tweede doelwit was die ‘DELTA nursing measure’ se bruikbaarheid vir verpleging bestudeer. Om dit te doen was ’n nuwe span psigiatriese verpleegkundiges (n = 25) wat werksaam was in ’n 116-bed akute psigiatriese hospitaal opgelei en getoets in die gebruik van die ‘DELTA nursing measure’. Na 30 maande is ’n fokusgroep (n = 6) gehou wat verteenwoordigend was van die span om hulle persepsies en ervarings van die nuut-ontwikkelde meetskaal se bruikbaarheid te eksploreer. Beskrywende data is met behulp van deduktiewe inhouds tegnieke geanaliseer. Die uitkoms van die ‘DELTA nursing measure’ as ’n routine verplegingmeetskaal vir die bepaling van akuutheid in psigiatrie het baie goeie resultate opgelewer. Die bruikbaarheidseienskappe het positiewe response bevestig dat aanvaarding, betroubaarheid en vertroue bereik is en dat dit ’n nuttige instrument is om die verplegingsdiens te bevorder. Die positiewe response op die ‘DELTA nursing measure’ is merkwaardig. Dit het die potensiaal om ’n betekenisvolle bydrae te lewer tot die psigiatriese verplegingsveld omdat dit ’n meetbare dimensie toevoeg tot pasient uitkomste, ’n hoogs-benodigde vereiste van pasiënte, multidissiplinêre spanne en gesondheidsorgbefondsers.


There is an ongoing global search for validated nursing measures regarding the severity of patients’ illness in the field of acute mental healthcare. The nursing profession requires a standardised method, similar to the vital signs, to observe, score and record the severity of a patient’s mental illness and to monitor it routinely as it changes from day to day. If such a method can be found and data can be collected routinely, empirical information on patient outcomes in acute mental-health care will be available to provide evidence for clinical effectiveness.

Scales currently competing to become a routine nursing measure are structured self-report questionnaires such as the Behavioural and Symptoms Identification System (BASIS) designed by Eisen and Dickey in 1996. The BASIS contains 32-items that capture the patient’s perspective on relation to self and others, depression and anxiety, daily living and role functioning, impulsive and addictive behaviour and psychosis. However, as a subjective questionnaire it has limited use in a country such as South Africa where levels of illiteracy remain high (Eisen & Dickey 1996:182).

Another competing scale that measures mental-health outcomes, according to Meagher et al. (2009:172) is the Health of the Nation Outcomes Scale (HoNOS), widely used in Britain, Australasia, Canada and some European countries. This is a 12-item scale measuring behaviour, impairment, symptoms and social functioning that can be completed by any member of the healthcare team. There seems to be little consensus in the literature to its usefulness as a routine measure and its accuracy with regard to detecting change. In fact, Bebbington et al. (1999:391) and Adams et al. (2000:196) argue against the use of scales such as HoNOS as a standardised routine measure as it has been found to have questionable validity and only a tenuous relationship with patient severity. Furthermore, Lakeman (2004:212) takes a strong nursing perspective in that clinician-rated standardised tools such as the HoNOS have little, if anything, to do with or to offer toward the patients’ recovery process, whilst Trauer, Callaly and Herrman (2009:294) report that the HoNOS enjoys limited acceptance by healthcare professionals as being a useful measure. Moreover, Meagher et al. (2009) caution that service needs cannot be judged merely upon a cross-sectional assessment of active symptomatology on admission and discharge as rendered by the HoNOS, but rather require routinely-observed and recorded longitudinal data on patient changes as these occur.

A third competitor is the Global Assessment Functioning (GAF) scale that indicates the severity of mental illness. It is used frequently by psychiatrists and other members of the healthcare team and is often referenced in research. It constitutes the fifth axis of the Diagnostic and Statistical Manual of Mental Disorders (DSM), currently undergoing its fifth review (Aas 2011). However, the GAF is not a measure, but rather a clinical classification based on the clinician’s judgement of the patient’s overall level of function (American Psychiatric Association 1994:199). The literature expresses significant concern about the GAF’s subjectivity, its lack of detail in its user guidelines and poor validation ratings (Aas 2010). In an attempt to satisfy these shortfalls, researchers and developers are currently trying to improve, adjust or repair the inadequacies of the GAF (Aas 2011).

Even with all these pressures to find a suitable nursing measure, Salvi, Leese and Slade (2005:146) reported little consensus as to which outcome scales to use in mental healthcare, and suggested that meaningful and comprehensive clinical information could only be provided by a combination of existing measures. However, Aas (2010) cautioned that if the numbers of scales are increased, the learning period for managing the scoring method may increase, scoring may become more time consuming and less easy to use and the outcomes analysis of the data may become more complex.

Lately, pressure from clinicians is mounting, challenging the randomised control studies of evidence-based practice as being too far removed from the real world of mental healthcare practice. Clinicians are advocating a parallel consideration for practice-based evidence as being more connected to the context of real practice. Furthermore, clinicians rate first-hand knowledge and experience of what works, what needs to be changed and how it may change due to experience of their patient-based outcomes, higher than the prescriptive formularies of academics (Hellerstein 2008; Warroll 2007). Irrespective of the ongoing debate between the preference for evidence-based practice or practice-based evidence, the point of consensus still requires the need for routine measurements of patient progress (APA Presidential Task Force on Evidence-Based Practice 2006:275).

Acute mental healthcare settings are also the domain of multidisciplinary teams and the question has been asked if the nursing profession is best suited, placed and skilled to measure patient progress on a routine basis. Meagher et al. (2009:173) reiterated the consensus that a multidisciplinary team is the preferred approach for mental healthcare services to record the complexities of severe mental illness. The continuous contact with patients and proximity of nurses as primary caregivers enable direct observations of changes as they occur. It has been well recorded that this advantage creates a rich intuitive knowledge integral to the nursing practice which Billay et al. (2007:149) emphasise as a legitimate form of nursing skill. In their line of duty, nurses already report their daily observations descriptively with regard to patient severity and progress. All these factors point toward nurses as being the best-skilled health professionals to observe, score and record patient outcomes routinely as changes occur. However, the existing scales as discussed above do not include the nursing profession as users and a literature review revealed no validated nursing framework available to nurses to allow them to observe and measure objectively the severity of patient mental illness.

Problem statement
Psychiatric nurses are at a disadvantage because they lack the tools to establish empirically the severity of their patients’ acute mental illness. Without a valid empirical nursing measure on which to base their nursing evaluation and assessment, the psychiatric nursing process and care plans are vulnerable to contradiction, inconsistency and ambiguity. Furthermore, the psychiatric nursing process currently has no empirical record of patient recovery and no routine longitudinal data on patient outcomes are available to reflect on the impact of nursing performance.

Aims and objectives of study
This study aims to provide psychiatric nurses with an objective scale to measure the severity of their patients’ acute mental illness. Two objectives were identified:

• To design and develop a nursing measure for psychiatric nurses to observe, score and record their patients’ severity of mental illness routinely as they recover by virtue of the acute psychiatric nursing process.

• To test if the newly-developed nursing measure has nursing utility, namely, to determine whether it is useful and acceptable enough to be embedded into the nursing process and care plans by psychiatric nurses.

Definition of key concepts
Scale versus measure: In general, the terms ‘scale’, ‘instrument’, ‘test’, ‘tool’ and ‘questionnaire’ may all refer to the concept of producing numerical scores to explain a phenomenon at an ordinal level (Bond & Fox 2007). However, none of these terms refer to a ‘measure’, which identifies a higher level of measurement qualities, namely linear interval characteristics (Stevens 1946).

Nursing utility: The utility of an instrument is the degree of conviction that the users have regarding its usefulness in their practice (Toomey, Nicholson & Carswell 1995), taking into consideration application practicalities such as relevance, suitability, feasibility, accuracy, comprehensiveness, credibility, flexibility, value and adaptability (Barbara & Whiteford 2005). The term clinical utility is often used, but for this study the term nursing utility has been used to fit the population being studied.

Psychiatric nurse: A category of nurses who obtained a qualification either through a four-year or a post-registration training programme and who is registered with the South African Nursing Council (SANC) as a psychiatric nurse. Countries such as the UK and Canada call this category of nurses mental-health nurses.

Mental health, mental illness and psychiatry: Mental health refers to cognitive, emotional and psychological wellbeing in order to function in society and meet the ordinary demands of everyday life. Mental healthcare is the branch of healthcare which provides care to those with impairments in cognitive, emotional or psychological areas. Mental illness is the term used to refer to impairments and dysfunction in areas of mental health. The term psychiatry is also found in literature. This term refers to the field of mental healthcare but seems to be used less in recent literature. The South African Mental Healthcare Act 17 of 2002 (South Africa 2004) uses the term mental healthcare and mental illness and these terms are used in this article. However, there is one exception. The term for nurses who are trained in mental healthcare will be psychiatric nurses as this is the category of nurses stipulated by the South African Nursing Council.

Research design and method

Research design
A qualitative, exploratory and sequential research design was followed to develop a new nursing measure and to test it for nursing utility. This qualitative study strove to understand the actions and processes within the specific context of psychiatric nursing and, explicitly, the interface between the patient and the psychiatric nurse. Exploratory research examines this interface actively as the specific phenomenon of interest rather than observing and reporting (Lobelo 2004:20). Exploratory research often includes the use of a panel of experts. The inclusion of experts in the design, development and testing of instruments have been advocated widely (Bruce, Langley & Tjale 2008:58; Burns & Grove 2005:400; Elo & Kyngäs 2007:108). The input from experts in this study was obtained through unstructured interviews and focus group interviews.

The research design was also sequential in nature as the findings of the first objective were used in the second objective. The nursing measure first had to be developed before it could be subjected to the test for nursing utility.

The sample
The sequential nature of the study necessitated two different samples. The sample for the first objective was a purposive sample of five psychiatric nurses. These nurses have been working as a team for more than four years in an acute psychiatric unit. The researcher was referred to this specific nursing team by the funding organisations when enquiring for a nursing service of excellence. The nurses in this team were valued for their excellence in mental healthcare and were thus viewed as experts by the lead researcher. The team agreed to meet as a group and thereafter made themselves available individually to edit the initial documentation of the newly-developed nursing measure. The sample size was determined by the availability of respondents and data saturation. When using experts in the development of an instrument, a sample of five to 10 participants is suggested (Burns and Grove 2005:407).

For the second objective of the study, a second sample of nurses was selected in order to test the newly-developed measure for its nursing utility. None of the nurses of the second sample were acquainted with the first group of nurses who participated in the design and development of the measure. A new sample was selected to control for possible bias from the first group when testing the measure for nursing utility. The participants of the second sample (n = 6) were selected from a population of 25 nurses employed at a 116-bed acute psychiatric facility in South Africa. This facility admits patients who are referred from a 72-hour unit as stipulated in the South African Mental Healthcare Act 17 of 2002 (South Africa 2004). The average length of stay is 23 days.

The inclusion criteria stipulated that participants should be registered psychiatric nurses, should have used the DELTA nursing measure for at least six months (implying that they were credentialed in the use of the measure), should have given consent to participate in the focus group and were scheduled for duty on the day of the focus group interviews. The sample consisted of five psychiatric nurses, one with a Master’s qualification in psychiatric nursing. The facility’s nursing manager was the sixth participant and, although she had no formal psychiatric qualification, her substantial experience in managing nursing services in this specific acute psychiatric facility made her input valuable. Each participant had between seven and 30 months’ experience in using the DELTA nursing measure.

Data collection and analysis
The design and development of the new nursing measure commenced with unstructured individual interviews with the nursing team in order to get an understanding of the behavioural symptoms and signs of patients with acute mental illness that nurses are able to observe and interpret. The lead researcher named the new measure the ‘DELTA nursing measure’. The name DELTA refers to the fourth letter in the Greek alphabet and is the fourth in a suite of nursing measures under construction by the researcher.

The unstructured interviews with the expert nurses were explorative and open-ended with the objective to explore their perceptions of what constitutes acuity in mental illness and how they would distinguish consistently between the levels of severity of an illness. Since the DELTA nursing measure must ultimately perform as a global nursing instrument of acute mental illness, special emphases were laid on the generalisations of observations; thus observations should be invariant for all diagnostic groups (observations to cover all patients with mental illness). After the five participants gave their input during the unstructured interviews, group sessions were held to discuss the observations collectively. The central questions used during the individual interviews remained: What do you see or observe when you know this patient is critically ill? How does severity present itself? How do you recognise the first signs of improvement? What are the observational behaviour changes that can serve as stepping stones to recovery and how do you distinguish between them? What observations do you experience as improvement and regression? Field notes, flip charts and audio recordings were made during the focus group sessions.

The team of expert nurses was also used to assist with the analysis of the raw data. This was achieved in two sessions with the expert team, scheduled at their facility and at their convenience. An inductive content analyses technique was used to reduce volumes of verbal- and printed descriptive material into more manageable data. The experts were asked to identify patterns from the raw data and explain their understanding of the phenomena of severity of mental illness. The inductive approach enabled the researcher to identify key themes in the area of interest by reducing the material to a domain, a set of themes or items and categories.

After the first inductive content analysis session, the researcher used the design structures proposed by the experts and created a documented first version of the DELTA nursing measure. This was sent to the experts for editing and during the second work session the team provided valuable information for the change, adjustment, modification and refinement of the content of the DELTA nursing measure. This became the second version of the measure. Throughout the analyses, the following four design guidelines were considered:

• The measure must be useful at the interface between psychiatric nurses and acute mentally-ill patients.
• The measure must fit the acute psychiatric nursing process and care plans.
• The measure scores must facilitate a uniform language in communicating acute mental illness.
• The measure must improve the quality of the acute psychiatric nursing services.

The guidelines above were adapted from Smart’s (2006:378) multidimensional model of clinical utility to create the transition to the second objective of the study. The experts agreed to implement the second version of the DELTA nursing measure in their facility to test for applicability and preliminary nursing utility. This was done for six months and during this time further minor changes were made, resulting in the formulation of the third version of the DELTA nursing measure. This third version was used to develop a training manual that included testing material to credential future users of the DELTA nursing measure.

Data collection and analysis in relation to the second objective of the study were ready to commence after the third version of the DELTA nursing measure was finalised. For this phase of the study, the team of 25 nurses in the 116-bed facility were credentialed in the use of the DELTA nursing measure. Each nurse underwent a six-hour training session and was subjected to a credentialing examination consisting of four case studies. They were given a certificate of competence on achieving an 80% pass mark. Nurses who failed were requested to repeat the test until they obtained at least the 80% pass mark. The nursing team scored all patients admitted into the facility on the DELTA nursing measure over a period of 30 months. Each patient had an admission score, weekly interval scores and a discharge score. During this period, the team recorded 9413 raw scores collected from 1955 patients admitted into the facility.

A focus group interview was held, after the 30-month period of using the DELTA nursing measure, with six participating nurses according to the inclusion criteria described above. The four broad questions selected for the focus group interview included the DELTA nursing measure’s overall usefulness to the nursing profession; the ease with which it could be embedded into the nursing process as a routine measure; its ability to become a universal language as measure of the severity of acute mental illness; and its potential to improve the quality of nursing care. These four questions related to the four design guidelines used during the second version of the DELTA nursing measure. The analysis thus followed a deductive content-analysis approach (Elo & Kyngäs 2007:111) where existing data were retested in a new context. The responses were then recorded and transcribed.

Ethical considerations

Prior to data collection, written approval was obtained from the clinical managers of the facilities where the data were collected, the Human Research Ethics Committee (Medical) of the University of the Witwatersrand (Ethical Clearance Certificate number M 10524) as well as the North West Provincial Department of Health.

Informed consent was obtained after explaining the purpose and procedure of the study. Nurses were informed regarding what was expected of them, that there were no known harm or risks in the study, that their participation was voluntary and that they could withdraw at any stage without being disadvantaged in any way. Nurses who agreed signed a consent form.


The four criteria of trustworthiness, namely credibility, dependability, confirmability and transferability were followed (Krefting 1991:214). Credibility was achieved by prolonged engagement with the participating nurses during the interviews and during development and testing procedures. The study was also examined by both internal- and external examiners and was published as part of a PhD dissertation (Loubser 2012). Dependability was achieved through a detailed description of the research methodology and peer review, as well as through use of a code-recode process during data analysis. Confirmability was achieved through a detailed description of the research process, unstructured individual and focus group interviews were recorded and field notes were written. Transferability was achieved through the purposeful selection of the sample, as well as through a dense description of the research methodology and results of the study, so that researchers who are interested in conducting similar research should be thoroughly informed.


First study: Design and development
Five latent variables that contributed equally to the severity of acute mental illness emerged from the individual- and group discussions with the five experts. These five variables became the five items of the DELTA nursing measure and were labelled as: acts of reality loss, acts of incongruence, acts of destructiveness, acts of self-absorption and acts of focus loss. Each of these five items embraces its own unique cluster of typical mental-illness terms and the definitions underpinning it (Table 1). For each item, a scale with seven categories (points on the scale) of severity was established (Figure 1). The first category (1 on the scale) represented the most acutely-ill patient with the least functionality and the seventh category (7 on the scale) represented the highest-functioning patient. Decision trees with guiding questions were developed for each item in order to arrive at the relevant category or score on the scale (Loubser 2012:234). Figure 2 is an example of a decision tree for one of the items, namely ‘the acts of reality loss’. The DELTA nursing measure thus consists of a domain (‘severity of acute psychiatric illness’) with five items, each having seven categories of severity as demonstrated in Figure 1. Therefore, when totalling the scores of the DELTA nursing measure, a minimum score of 5 (5 items multiplied by the minimum score of 1) can be achieved in a critically-ill patient and a maximum score of 35 (5 items multiplied by the maximum score of 7) would indicate a normal functioning person.

TABLE 1: Items of the DELTA nursing measure with definitions.

FIGURE 1: Radar graph representing the DELTA scale structure.

FIGURE 2: Decision tree for acts of reality loss to obtain a score.

Second Study: Nursing utility
The findings below derived from a focus group interview with six nurses responding to the four guiding questions on the nursing utility of the DELTA nursing measure. In an attempt to convey the richness of the data as expressed in the language used by the participants in the group, any direct quotations used must be interpreted as coming from the spokesperson of the group when expressing an experience or opinion.

Question 1: Is the DELTA nursing measure useful to the nurses?
Overall the nurses reported a ‘new mind-set change’ toward severity measurement during the nursing assessment and evaluation. For the novice nurse with little or no previous mental healthcare experience, the inclusion of this new vital sign in the nursing process was initially difficult to master, but ‘within months’ the DELTA nursing measure became ‘very easy to use’. Once proficient, the nurses reported new insights in their practice as they found themselves able not only to measure ‘if the patient is improving or not improving’, but also to evaluate and record ‘how much the patient is improving from admission to discharge’. This new nursing insight has empowered them to guide the multidisciplinary team as their nursing statistic became ‘a good indicator to see if things start to go wrong for the patient’. As the nurses started to use it routinely, they also found themselves to be ‘very successful to further evaluate [sic] deterioration or improvement in patient scores and even adjust treatment according to the scores’. Overall, the nurses reported a useful new vital statistic whereby they found ‘it very helpful to be able to evaluate [empirically] if the patient is improving or not improving’. They also felt that the scores of the DELTA nursing measure correlated with their clinical judgement regarding the severity of symptoms. This enhanced their confidence in the scores to the extent that they advocate adjusting treatment and services based on their recordings. (See Table 2)

TABLE 2: Summary of findings of the focus group interview on nursing utility.

Question 2: Can it be embedded into the nursing process?
All responses indicated that the nurses used the DELTA nursing measure routinely during the assessment, diagnosis, implementation and evaluation phases of the nursing process and found it to be ‘beneficial to the nursing process’. They provided evidence showing how the DELTA nursing measure ‘mind-set’ helped their nursing process to focus on the five items in the DELTA nursing measure. ‘Previously’, they reported, ‘it was difficult to interview, but if you know the content of the DELTA nursing measure it makes it now much easier to interview as we now know what to look out for’. Moreover, ‘it makes it easier to make a nursing diagnosis and establish the severity of the diagnosis’. The nurses also highlighted four awareness features of the DELTA nursing measure, which have become embedded into the routine nursing process: firstly, the DELTA nursing measure prompted nurses to identify specific symptoms which directed them to the appropriate nursing diagnosis; secondly, the rating of the severity led them to devise and implement appropriate nursing care plans; thirdly, the DELTA nursing measure has the ability to raise warnings on suicidal risks; and, finally, the longitudinal scoring and recording of patient severity are used to advocate for treatment changes at the multidisciplinary team meetings.

Question 3: Can the items of the DELTA nursing measure be used as a uniform language?
As the DELTA nursing measure becomes embedded into the routine nursing process, changes in the scores become helpful references of communication between nurses and doctors to allow for the discussion of diagnosis and treatment plans. The DELTA nursing measure became the universal language at the multidisciplinary meetings. According to the nurses, the multidisciplinary teams are becoming more reliant on the feedback from the scores on the DELTA nursing measure as the preferred outcome measure to evaluate patient progress or decline. Nurses find it an ‘exciting new development’ that multidisciplinary teams base their considerations regarding adjustment of treatment plans more on the nursing observations than ever before. Nurses further emphasised their ability to score a patient empirically as a suicide risk and used this score as motivation for management to supply such patients with one-to-one nursing care.

Question 4: Does the DELTA nursing measure improve the quality of nursing services?
Whilst probing the issue of nursing quality of care, there was consensus that the DELTA nursing measure ‘definitely made nurses more aware of their patient’s symptoms, needs and severity of illness’. As the DELTA nursing measure became the universal language at the multidisciplinary meetings, nurses became more confident in defending their scores and patient outcomes. There was also a vast improvement in their patient-interaction skills, interviewing skills and their observations of patient behaviours. Being more aware and knowing the patient better also resulted ‘in rendering better conditions of care’. Thus, the DELTA nursing measure, being a dedicated nursing measure and producing scores to calculate patient outcomes, not only ‘contributes to improving the quality of nursing care’, but also contributes to ‘improving the nursing skills’.

It became evident that nurses regularly discussed the scores of the DELTA nursing measure amongst themselves so as to evaluate their nursing process. A decline in scores indicated a regression that necessitates re-evaluation of nursing interventions and care plans whilst increased scores indicated improvement that may or may not require adjustments to treatment. One nurse described these discussions as an ‘exciting experience’.

The findings from the focus group interview supported the nursing utility of the DELTA nursing measure as a clinically-useful measure of severity of acute mental illness. No signs of hesitation by the nurses in any of the four questions to express a positive response were noted by the researcher, thus indicating that the DELTA nursing measure was well accepted by the nursing team in this study.


Psychiatric nurses working in acute mental healthcare facilities are at a disadvantage as they lack the measurement tools to track change after intervention. As a result, they are unable to monitor empirically a patient’s progress or to implement appropriate care plans. Although a systematic review of interventions delivered by UK mental-health nurses revealed that nurses are able to deliver effective interventions, the authors reported that a limitation of the study was that a precise assessment of outcomes was not possible. The reason for this is the lack of appropriate and valid measurement of outcomes that are sensitive to change and specific to the situations concerned (Curran & Brooker 2007:501). Montgomery, Rose and Carter (2009:32) reviewed the literature to examine the relationship between psychiatric mental-health nursing interventions and patient-focused outcomes. Their most important finding was that there are inconsistent recommendations as to which instruments should be used routinely in practice. The introduction of the DELTA nursing measure into the scope of psychiatric nursing in South Africa generated a new awareness that nurses should focus on observations and measurement of severity of symptoms in acute mentally-ill patients and the implementation of routine outcome measurement. The design and development of the DELTA nursing measure were dependent on the input from expert nurses who contributed to the positive responses toward nursing utility as well as the appropriateness of the content of this measure to the specific context of acute mental illness in South Africa.

With no previous training or information given on acute mental healthcare nursing techniques to improve patient outcomes, it is interesting to note that the psychiatric nurses discovered by themselves that they ought to change the focus of their scope of practice from a reactive task-orientated approach to proactive patient advocacy in order to achieve both patient and nurse satisfaction. The availability of empirical DELTA scores has widened the nursing approach from basic task rendering to an all-inclusive nursing approach with an emphasis on patient-focused outcomes.

Data for the development of the items were generated from a small sample of psychiatric nurses (n = 5), which might be seen as a limitation of the study. However, the feeling is that the larger the number of experts, the less consensus is reached (Burns & Grove 2005:407). Working with this small number of people in one institution had the advantage that participants could share all their ideas with fewer interruptions in the group. The institution was also rated by a funding organisation as being one of the settings that provided a good service, allowing the authors to assume that the nursing team are experts in their field and thus able to share rich and relevant information.

There is a paucity of literature on nursing measures applied routinely for the measurement of the severity of mental illness. This eliminated the possibility to review other measures and to compare items that emerge from this study to previous studies. It seems that nurses working in the mental healthcare field either form part of multidisciplinary outcome measurements such as HoNOS (Brooks 2000:506) or specific research that does not entail routine outcome measurement, for example, the GAF (Skärsäter, Baigi & Haglund 2006:206) and the Brief Psychiatric Rating scale (Myer et al. 2008:147). None of these measures have been developed with nurses and with the focus on acute mental illness. This may limit their credibility amongst nurses and subsequently decrease their motivation to implement such measures routinely. The newly-developed DELTA nursing measure is intended for routine outcome measurement in acute mentally-ill patients, developed with input from psychiatric nurses. There seems to be an optimism that it fits into the nursing process.


The paucity of published work on routine outcome measurement in nursing for patients with mental illness limits comparison with other measures (Curran & Brooker 2007:501; Montgomery et al. 2009:32). Limited publications in routine outcome measurement might not necessarily point to an absence of relevant measures and therefore the authors would welcome feedback on outcome measures for nursing in mental healthcare settings.

The sample was selected from one institution and was thus not representative of the entire country. Although quality data were obtained, bigger samples could support relevance and content validity of the items for the DELTA nursing measure. Research is underway to determine the construct validity of the DELTA nursing measure and preliminary results indicate good validity (Loubser 2012:257). These results could be an indication that, in spite of the limited sample of five experts, the items are indeed valid for South African acute mentally-ill patients.


The findings of this study suggest that the psychiatric nursing process could benefit from a ‘mind-set change’ (as the sample in this study experienced) to include sustainable procedures to maintain the nursing focus on patient outcomes. This study also revealed that the DELTA nursing measure is an important instrument to facilitate the process of evidence-based practice.

Future nursing education may wish to consider including training, testing and credentialing on the use of the DELTA nursing measure for focused nursing-care plans and improved quality in acute mental healthcare.


The DELTA nursing measure was developed by South African nurses for South African nurses in order to render better care for patients requiring acute mental healthcare. The findings from this study supported the DELTA nursing measure as a well-designed measure that can be used routinely by psychiatric nurses in acute mental healthcare facilities to provide empirical evidence of their patient’s severity of mental illness. It has made nurses aware of the value of having empirical patient-based outcomes data embedded into the nursing process. This was manifested explicitly in the ease with which the nurses recognise, observe and recorded the scores of the DELTA nursing measure in their daily routine. Psychiatric nurses in this sample reported positive outcomes such as an improvement in the quality of nursing services and a uniform language to communicate with team members. In this process the nurses also became aware instinctively that they are the advocates of their patients’ recovery process and this put them back in control of nursing effectiveness and efficiency. The DELTA nursing measure adds a measurable dimension to patients’ mental health outcomes, a much-needed requirement by patients, multidisciplinary teams and healthcare funders.


The authors would like to acknowledge the nurses that participated voluntarily in the study, as well as our colleagues from the School of Therapeutic Sciences at the University of Witwatersrand for their support as peer reviewers in the study, most specifically our colleague, Dr Marietha Nel, for her scientific- and language editing.

Competing interests
The South African Database for Functional Measures (Pty) Ltd provided financial support for this study. Although the SADFM gave written permission to publish this article, and although the copyright of this article vests with the publishers, the patent rights of the DELTA vests within SADFM (RSA patent registration number 2008/09086).

Authors’ contributions
This article was based on the primary research done by H.J.L. (Wits University) under supervision of J.C.B. and D.C. (Wits University) for the PhD thesis: ‘The validation of nursing measures for patients with unpredictable outcomes’. The draft was formulated by H.J.L. and J.C.B. and D.C. contributed to the finalisation of the article.


Aas, I.H.M., 2010, ‘Global Assessment of Functioning (GAF): Properties and frontier of current knowledge’, Annals of General Psychiatry 9, 20.

Aas, I.H.M., 2011, ‘Guidelines for rating Global Assessment of Functioning (GAF)’, Annals of General Psychiatry, 10, 2.

Adams, M., Palmer, A., O’Brien, J.T. & Crook, W., 2000, ‘Health of the Nation Outcome Scales for psychiatry: Are they valid?’, Journal of Mental Health 9(2), 193–198.

American Psychiatric Association (APA), 1994, Diagnostic and statistical manual of mental disorders, 4th edn., American Psychiatric Association, Washington, DC.

APA Presidential Task Force on Evidence-Based Practice, 2006, ‘Report of the 2005 Presidential Task Force on Evidence-Based Practice’, American Psychologist 61, 271–285. PMid:16719673

Barbara, A. & Whiteford, G., 2005, ‘Clinical utility of the Handicap Assessment and Resource Tool: An investigation of its use with the aged people in hospital’, Australian Occupational Therapy Journal 52(1), 17–25.

Bebbington, P., Brugha, T., Marsden, L. & Window, S., 1999, ‘Validation of the Health of the Nation Outcomes Scale’, The British Journal of Psychiatry 174, 389–394., PMid:10616602

Billay, D., Myrick, F., Luhanga, F. & Yonge, O., 2007, ‘A pragmatic view of intuitive knowledge in nursing practice’, Nursing Forum 42(3), 147–155., PMid:17661807

Bond, T.G. & Fox, C.M., 2007, Applying the Rasch Model: fundamental measurement in the human sciences, Lawrence Erlbaum Associates, Mahwah, NJ.

Brooks, R., 2000, ‘The reliability and validity of the Health of the Nation Outcome Scales: validation in relation to patient derived measures’, Australian and New Zealand Journal of Psychiatry 34(3), 504–511. PMid:10881976

Bruce, J.C., Langley, G.C. & Tjale, A.A., 2008, ‘The use of experts and their judgments in nursing research: An overview’, Curationis 31(4), 57–61., PMid:19653538

Burns, N. & Grove, S.K., 2005, The practice of nursing research: Conduct, critique and utilization, W.B. Saunders Co, New York.

Curran, J. & Brooker C., 2007, ‘Systematic review of interventions delivered by UK mental health nurses’, International Journal of Nursing Studies 44(3), 479–509., PMid:17241635

Eisen, S.V. & Dickey, V., 1996, ‘Mental health outcomes assessment: the new agenda’, Psychotherapy 33(2), 181–189.

Elo, S. & Kyngäs, H., 2007, ‘The qualitative content analysis process’, Journal of Advanced Nursing 62(1), 107–115., PMid:18352969

Hellerstein, D.J., 2008, ‘Practice-based evidence rather than evidence-based practice in psychiatry’, Medscape Journal of Medicine 10(6), 141, viewed 18 August 2011, from

Krefting, L., 1991, ‘Rigor in qualitative research: the assessment of trustworthiness’, American Journal of Occupational Therapy 45(3), 214–222., PMid:2031523

Lakeman, R., 2004, ‘Standardized routine outcome measurement: pot holes in the road to recovery’, International Journal of Mental Health Nursing 13(4), 210–215., PMid:15660588

Lobelo, M.I., 2004, ‘Experiences of relapsed psychiatric patients in Mafeking in the North West Province’, MCur Thesis, Dept. of Nursing Sciences, Rand Afrikaans University, Johannesburg.

Loubser, H.J., 2012, ‘The validation of nursing measures for patients with unpredictable outcomes’, PhD Thesis, Dept. of Nursing Education, University of the Witwatersrand, Johannesburg.

Meagher, D., O’Brien, S., Pullela, A., Oshun, A. & Brosnan, P., 2009, ‘Multidisciplinary activities in a community mental health service: Relationship to Health of the Nation Outcomes Scale scores and diagnosis’, The Psychiatrist 33(5), 172–175.

Montgomery, P., Rose, D. & Carter, L., 2009, ‘Patient health outcomes in psychiatric mental health nursing’, Journal of Psychiatric Mental Health Nursing 16(1), 32–45., PMid:19192084

Myer, L., Smit, J., Roux, L.L., Parker, S., Stein, D.J. & Seedat, S., 2008, ‘Common mental disorders among HIV-infected individuals in South Africa: prevalence, predictors, and validation of brief psychiatric rating scales’, AIDS Patient Care and STDs 22(2), 147–158., PMid:18260806

Salvi, G., Leese, M. & Slade, M., 2005, ‘Routine use of mental health outcome assessments: choosing the measure’, The British Journal of Psychiatry 186, 146–152., PMid:15684239

Skärsäter, I., Baigi, A. & Haglund, L., 2006, ‘Functional status and quality of life in patients with first-episode major depression’, Journal of Psychiatry and Mental Health Nursing 13(2), 204–213., PMid:16608476

South Africa, 2004, Mental Health Care Act No 17 of 2002, Government Printer, Pretoria.

Stevens, S.S., 1946, ‘On the theory of scales of measurement’, Science 103(2684), 677–680., PMid:17750512

Smart, A., 2006, ‘A multi-dimensional model of clinical utility’, International Journal for Quality in Health Care 18(5), 377–382., PMid:16951425

Toomey, M., Nicholson, D. & Carswell, A., 1995, ‘The clinical utility of the Canadian Occupational Performance Measure’, Canadian Journal of Occupational Therapy 62(5), 242–249.

Trauer, T., Callaly, T. & Herrman, H., 2009, ‘Attitudes of mental health staff to routine outcome measurement’, Journal of Mental Health 18(4), 288−297.

Worrall, J., 2007, ‘Why there’s no cause to randomize’, British Journal of the Philosophy of Science 58(3), 451–488.

Crossref Citations

No related citations found.