Nurses’ Evaluations of the Feasibility and the Clinical Utility of the Critical-Care Pain Observation Tool

Background 

A consensus has not yet been reached regarding the definitions of feasibility and clinical utility; as such, these concepts are sometimes used interchangeably (Stevens & Gibbins, 2002). Generally, feasibility refers to the ease with which the clinicians can apply the tool in the clinical setting (e.g., the tool is simple to understand, easy to complete, and quick to use) whereas clinical utility refers to the ability to use the results of the tool in a useful or informative way within the clinical setting (Duhn & Medves, 2004). Although reliability and validity testing of critical care pain assessment tools is well underway, to date the feasibility and clinical utility of only the PAIN algorithm (Puntillo, Stannard, Miaskowski, Kehrle, & Gleeson, 2002) and the BPS (Payen et al., 2001) have been explored.

The PAIN algorithm is a systematic pain assessment and management tool developed by a team of critical care nurses and pain experts (Puntillo et al., 2002). The first step consists of a checklist (present or absent) and a numeric rating scale (0 to 10) which prompts the nurse to assess whether or not the patient has pain. The checklist contains 20 items clustered in a behavioral dimension (movements, vocalization, facial indicators, and posturing/guarding) and a physiologic dimension (heart rate, blood pressure, respiratory rate, perspiration, and pallor), the latter being based on the nurse's clinical judgment. The pain assessment part of the PAIN was tested with 31 critically ill postoperative adult patients, most of them extubated (Puntillo et al., 1997). It showed acceptable criterion validity when the number of behavioral and physiologic indicators observed were compared with the nurse's ratings of pain intensity (r = 0.17 to 0.77; p < .05). The second and third steps of the algorithm involve assessing for potential problems influencing opioid administration and making an analgesic treatment decision. Of the eleven nurse evaluators who used the PAIN, ten found the instrument to be helpful (Puntillo et al., 2002). Their comments indicate that this was because it added structure to their practice and increased their awareness of pain indicators. Four nurses felt the instrument could be improved, and their comments speak to the need for reducing the length and complexity of the tool. Five nurses believed the algorithm changed their practice, and six felt that it would have a lasting effect on their practice. Ten nurses believed that they would have benefited from having the tool earlier in their practice, and eight felt that other nurses on the unit could benefit from it. In summary, although the PAIN appears to be a useful tool to standardize pain assessment and management in the ICU, some limitations have been highlighted. Namely, it does not standardize the measurements of behavioral and physiologic indicators, which are based on the nurse's clinical judgment, and the length of the tool limits its feasibility and clinical utility (Li, Puntillo, & Miaskowski, 2008).

The BPS was created to evaluate pain in critically ill sedated unconscious patients receiving mechanical ventilation (Payen et al., 2001). It takes into consideration three behavioral dimensions: facial expression, movement/positioning of upper limbs, and compliance with ventilation. Each behavior is scored from 1 to 4, for a possible total score ranging from 3 to 12. It was tested with three ICU mechanically ventilated and sedated adult samples: 30 trauma or postoperative patients (Payen et al., 2001), 30 medical patients (Aïssaoui, Zeggwagh, Zekraoui, Abidi, & Abouqal, 2005), and 44 medical or postoperative patients (Young, Siffleet, Nikoletti, & Shaw, 2006). All patients were unconscious and thus unable to self-report. Interrater reliability was high in the first two studies, with a weighted kappa coefficient of 0.74 (Payen et al., 2001) and an intraclass correlation coefficient of 0.95 (Aïssaoui et al., 2005), respectively. However, interrater agreement varied from low to high (36% to 91%) in the study by Young, Siffleet, Nikoletti, and Shaw, (2006), where lower agreement was obtained during pain assessments of nociceptive procedures. Discriminant validity was supported in all three studies where higher BPS scores were obtained with nociceptive procedures compared with rest or nonnociceptive procedures. Although feasibility and clinical utility were not formally evaluated, 28 nurses and nurse's aides completed a questionnaire for satisfaction and remarks (Payen et al., 2001). Twenty-four evaluators were satisfied with the instrument's ease of use, and all of them agreed that the assessment required only minimal time (2-5 minutes). Twenty-five felt that effective pain reactions had been assessed, and 26 expected changes in pain management as a result of the BPS (Payen et al., 2001). Seven evaluators expressed concerns regarding complexity of the BPS, although reasons for this were not specified.

Experts strongly recommend that a behavioral pain scale be used to support pain assessment in unconscious or nonverbal critically ill patients (Herr et al., 2006; Jacobi, Fraser, Coursin, Riker, Fontaine, Wittbrodt, et al., 2002). In fact, combining pain assessment with a standardized approach of pain management can lead to better pain control (Chanques, Jaber, Barbotte, Violet, Sebbane, et al., 2006). Knowing this, it is relevant to have access to validated pain tools as well as information about essential clinical characteristics, such as feasibility and clinical utility, to make the best decision about the selection of a tool for implementation into practice. From the available tools for pain assessment in critically ill nonverbal patients, the BPS and the CPOT are recommended in critical reviews (Li et al., 2008; Sessler, Grap, & Ramsay, 2008). Although it is mentioned that further testing is warranted for those two scales, the ambiguity of some of the items of the BPS is highlighted as well as some weaknesses in the validation process and results (e.g., results based on total number of observations rather than individual patients, and reliability results variable). Moreover, feasibility and clinical utility of the BPS have not been described in detail, and little information has been published regarding the methods and instrument used.

Aim 

The purpose of this study was to describe nurses’ evaluations of the feasibility and clinical utility of the Critical-Care Pain Observation Tool in assessing pain in critically ill ventilated adults.

Methods 

Results 

Nurse Participants 

Sociodemographic information of nurse participants is described in Table 1. The ICU nurses were aged 24-53 years, with a mean age of 39 years. Most of them were full-time staff nurses and had either a college diploma or a bachelor's degree in nursing. Years of experience in the ICU varied from 6 months to 28 years, with an average of 9 years. Two-thirds of them acknowledged that they had received education in pain in the past at school (college, university), at a conference, or at the hospital.

Table 1. Description of the Nurses Who Completed the Questionnaire (n = 33)

	Variable
	Age∗	38.9 ± 9.4
	Years of experience as an ICU nurse	8.9 ± 8.6
	Years of experience as a nurse	14.1 ± 9.8
	Gender
	Female	27 (81.8%)
	Male	6 (18.2%)
	Highest education level completed
	College diploma (CEGEP)	13 (39.4%)
	Certificate (university)	1 (3%)
	Bachelor's degree in nursing	18 (54.5%)
	Master's degree in nursing	1 (3%)
	Position in the ICU
	Staff nurse	28 (84.8%)
	Assistant nurse manager	4 (12.1%)
	Clinical educator	1 (3%)
	Employee status
	Permanent full time	28 (84.8%)
	Permanent part time	5 (15.2%)
	Have received education in pain	22 (66.7%)

Values are mean ± SD or n (%).

∗ This variable has one missing datum.

CPOT Feasibility and Clinical Utility 

Results from the evaluation form about the feasibility and clinical utility of the CPOT are presented in Table 2, and qualitative comments are described in Table 3. Overall, the feasibility of the CPOT was positively evaluated by the nurse participants. Most questions obtained 90%-100% 3 (sufficiently) and 4 (very) responses, except for question 3 about how quick the CPOT was to use, which reached <80% 3 or 4 responses. Three participants highlighted that the delay between the training and the actual use of the CPOT was too long, and two participants mentioned that the CPOT got quicker to use after getting familiar with it. For a few participants, the CPOT was more difficult to understand regarding choosing the right scoring from the 0 to 2 scale. One participant suggested reducing the number of choices available in the CPOT to make it easier to complete.

Table 2. Nurses’ Responses to the Questionnaire About the Feasibility and Clinical Utility of the Critical-Care Pain Observation Tool (CPOT)

		Frequency (n)
		1	2	3	4
	Question	Not At All	A Little	Sufficiently	Very	% of Nurses Who Answered 3 or 4
	Feasibility
	1. Was the length of time sufficient to train to use the CPOT accurately?	1	2	23	7	90.9%
	2. Were the directives about the use of the CPOT clear?	0	0	24	9	100%
	3. Is the CPOT quick to use?	1	2	20	6	78.8%
	4. Is the CPOT simple to understand?	0	0	23	10	100%
	5. Is the CPOT easy to complete?	0	2	16	15	93.9%
	Clinical Utility
	6. Would you recommend to use the CPOT routinely?	2	7	14	10	72.7%
	7. Is the CPOT helpful for nursing practice?	2	7	14	10	72.7%
	8. Has the CPOT influenced your practice in assessing the patient's pain?∗	3	11	11	7	54.5%

∗ This question has one missing datum.

Table 3. Nurses’ Comments About the Feasibility and Clinical Utility of the Critical-Care Pain Observation Tool (CPOT)

	Comment	Frequency
	Length of training to use the CPOT
	Explanation helped; the use of the CPOT became clearer once done at bedside	2
	Directives about the use of the CPOT
	Delay between the introduction of the study and the actual use of the CPOT was long; review of the directives would help before the first use of the CPOT and before the implementation of the tool into clinical practice	3
	The CPOT is quick to use
	The CPOT got quicker to use after getting familiar with it.	2
	The CPOT is simple to understand
	It was difficult to make out the difference between the choices (scale from 0 to 2) of some indicators of the CPOT (e.g. facial expressions, body movements)	2
	The CPOT is easy to complete
	Fewer choices needed	1
	Reasons for recommending the use of the CPOT routinely
	More uniformity among nurses, common language, organized and standardized way to assess pain in nonverbal patients; helps to validate our observations to the doctors	7
	It is important to determine pain in critically ill patients and treat it accordingly.	4
	The CPOT is a good tool to assess pain on a continuous basis in nonverbal patients.	3
	Other reasons (e.g., for special cases only, to improve our way of providing care)	2
	Reasons for not recommending the use of the CPOT
	Some indicators of the CPOT don't seem to be specific to pain (e.g., agitation, muscle rigidity)	2
	Extra tool targeting something that is already taken care of by the nurses	2
	The CPOT is patient selective; cannot be used in patients receiving neuromuscular blocking agents, deeply sedated	2
	The CPOT is helpful for practice
	The CPOT offers a standardized way of assessing pain in nonverbal patients and communicating with the care team	9
	Assessing and managing pain is an important aspect of practice	4
	It reminds us to be more aware of how procedures such as turning affect our patient's comfort	1
	The CPOT is not helpful for practice
	Complicated, extra tool, not practical	4
	Using the CPOT in the study has influenced my practice in assessing the patient's pain
	Helps to remind and be more sensitive to nonverbal cues of pain	9
	Helps to assess pain control	2
	Advocate for the patient's pain	1
	Using the CPOT in the study has not influenced my practice in assessing the patient's pain
	I've been looking for nonverbal cues of pain before the introduction of the CPOT	6
	Nothing compares with clinical judgment	1

According to its clinical utility, a little more than 70% of the nurses mentioned that the CPOT was helpful for nursing practice and recommended its use routinely. Nurses who recommended its use and mentioned its relevance for nursing practice acknowledged that the CPOT would allow more uniformity among nurses, because it provides common language and a standardized way to assess patients’ pain. The lack of specificity of some indicators of the CPOT and the fact that the tool could not be used with every ICU nonverbal patient were mentioned by nurses who did not recommend its use. Some nurses mentioned that the CPOT was an extra tool which was not necessary, complicated, or not practical to implement in routine care. Half of the nurse participants supported that the CPOT had influenced their practice in assessing the patient's pain. They mentioned that the CPOT helped them to remind and to be more sensitive to nonverbal cues of the presence of pain. For some nurses, the CPOT had not influenced their practice, because they were already sensitive to nonverbal cues of pain before the introduction of the tool. One nurse also highlighted that a tool should not replace clinical judgment.

Discussion 

The feasibility and clinical utility of the CPOT were examined for the first time in the present study. A high proportion of the nurse participants found the CPOT to be easy to use and relevant for clinical practice. Only 15% of the nurses (5 out of 33) criticized the CPOT as long or complex to use, compared with 36% of the nurses (4 out of 11) who used the PAIN tool (Puntillo et al., 2002) and 25% of those (7 out of 28) who used the BPS (Payen et al., 2001). The CPOT includes only four behavioral indicators, and the PAIN tool is a checklist of 20 items of behavioral and physiologic indicators. And the BPS, even though it includes only three behaviors, they are rated on a 4-point scale in which scores are not clearly described. The ambiguity of some of the BPS items as well as the difficulty in interpreting the tool score have been highlighted in recent critical reviews (Bjoro & Herr, 2008; Li et al., 2008). It has been suggested that the clinical utility of a pain assessment tool can be enhanced by including operational definitions for each indicator and score (Li et al., 2008). The CPOT includes operational definitions, as described in the Appendix.

However, some criticisms of the nurse participants of the CPOT need to be addressed. First, some indicators of the CPOT, such as body movements, may lack specificity to pain. A score of 0 is attributed when the patient is not moving or immobile. However, the absence of movements may not necessarily mean an absence of pain. Indeed, nurses have identified the absence of movements as an indicator of the presence of pain in ICU patients (Puntillo et al., 1997). Remaining immobile may be effective in protecting themselves from pain caused by movements. Also, body movements may vary depending on the patient's level of consciousness or physical factors (e.g., use of physical restraints, medications). For example, it has been reported that the higher the dosage of sedative agents, the lower was the score of a behavioral pain tool (Gélinas & Johnston, 2007; Payen et al., 2001), which makes the CPOT patient selective, as mentioned by two nurses. Unconscious patients are also more likely to exhibit fewer pain behaviors compared with conscious patients (Gélinas & Arbour, in press; Gélinas & Johnston, 2007). Moreover, experts suggest that body movement descriptors would be enhanced by categorizing these responses according to the patient's level of consciousness (Li et al., 2008). In addition, facial expressions seem to differ in head injury patients (Gélinas & Arbour, in press). Further research is warranted in this field to establish the specificity of behavioral parameters as a pain response in ICU patients. In the meantime, it must be emphasized that the CPOT should be used in nonverbal patients who have intact motor function so that they can respond to the tool's requisite behaviors. For instance, the use of the CPOT is not recommended with paralyzed patients and those who are receiving neuromuscular blocking agents. Also, low CPOT scores in heavily sedated patients may not necessarily reflect decreased pain level or the absence of pain in this group and should be interpreted with caution.

It was interesting that some nurses pointed out that they were already sensitive to nonverbal cues of pain before the introduction of the CPOT. This finding is a bit different from the study of Puntillo et al. (2002) 7 years ago with the PAIN tool. With the growing body of knowledge in the field of pain assessment in nonverbal patients in the last decade, nurses have been sensitized to the importance of identifying nonverbal cues of the presence of pain in their practice. Still, for the majority of them, the use of a standardized tool such as the CPOT was helpful. This is consistent with previous studies in which it has been found that the use of a standardized pain documentation tool can lead to better pain assessment (Voight & Paice, 1995) and that combining pain assessment with a standardized approach of pain management can lead to better pain control (Chanques et al., 2006). Knowing this, it is urgent that clinical guidelines and recommendations in pain assessment and management be implemented into practice (Jacobi et al., 2002; Herr et al., 2006).

Even if the use of behavioral scales is known as a valid approach to pain assessment, it must be emphasized that they are not always accurate reflections of pain intensity. Experts underscore that behavioral scores should not be interpreted as pain intensity scores nor compared with standard categories of pain levels (mild, moderate, severe). Results from a previous study also support this statement. Indeed, the CPOT was found to adequately classify postoperative ICU patients with severe pain but not those with mild or moderate pain (Gélinas et al., 2009). Thus, a behavioral score may be helpful in identifying the presence of pain (rather than rating pain intensity) and in evaluating interventions for pain management (Herr et al., 2006).

Conclusions 

So far, the CPOT has demonstrated acceptable results of reliability and validity in both conscious and unconscious ICU adults. Its feasibility and clinical utility have also been positively evaluated by ICU nurses who used it for the purpose of research. The use of a valid behavioral pain scale is recommended in clinical guidelines for pain assessment in nonverbal patients (Herr et al., 2006), and the CPOT is suggested by many experts in critical reviews (Li et al., 2008; Sessler et al., 2008). At this point, the CPOT is being used for either research or clinical purposes in many health care centers in Canada, the United States, and other countries. It is available in French and English, and translations in other languages are in progress (i.e., Finnish, Portuguese, Italian, and Swedish). Even if further research is warranted to improve the specificity of some indicators of the CPOT and to evaluate its feasibility and clinical utility in routine care, the CPOT appears to support ICU nurses in the assessment of their patients’ pain and could contribute to better pain control in the critically ill adult.