Measuring patient satisfaction with vascular access: vascular access questionnaire development and reliability testing

Introduction

Fistulas are associated with the lowest morbidity and mortality of all types of vascular access if they mature enough to be used to deliver adequate dialysis. There is clear clinical and economic data associating catheter use, especially catheter-related infections, with increased morbidity, rising hospitalization rates, high treatment costs and poor survival compared to use of a fistula or graft (1-2-3-4-5-6-7-8-9). However, high levels of catheter use persist, with up to 80% of incident patients initiating hemodialysis with a catheter and up to 50% of prevalent hemodialysis patients dialyzing via a catheter in North America (10, 11). Measuring the patient perspective on vascular access is critical to understanding the motives of patients in their vascular access choice.

When seeking to capture the patient’s perspective on vascular access it is critical to select an appropriate measurement tool. There are over 20 generic quality of life instruments that have been used in the hemodialysis population (12). The most commonly used are the SF-36 (13), and five preference-based generic quality of life measures, including the Health Utilities Index III (HUI3) (14, 15) and Euro-QOL (16, 17). However, general quality of life measures may not be appropriate to ascertain patients’ views on specific hemodialysis-related factors and processes, such as vascular access care, as they are too broad spectrum, containing very few, if any, items related to vascular access. In order to address this, there is a measure of hemodialysis patient satisfaction with their vascular access, called the vascular access questionnaire, developed by Quinn et al. The vascular access questionnaire is a direct estimation rating scale that estimates the level of hemodialysis patients’ satisfaction with their vascular access (18). The concept of measuring patient satisfaction, rather than “quality of life,” is practical and appropriate in the context of the vascular access care program evaluation. In a previous survey of 274 consecutive hemodialysis patients at three dialysis units, which aimed to measure patient adherence to the prescribed hemodialysis catheter dressing protocol and its impact on patient’s quality of life, we found that if patient satisfaction with their vascular access care protocol can be improved they are likely to be more compliant with it (19). The preexisting version of the vascular access questionnaire has not been rigorously tested or validated (18). In this study, the focus is the further development of the original vascular access questionnaire and psychometric evaluation of the resultant newer short-form vascular access questionnaire (SF-VAQ) for use in vascular access care programs.

Materials and Methods

Questionnaire administration

This project was undertaken as a quality assurance project at the University Health Network-Toronto General Hospital (UHN-TGH) to better understand hemodialysis patients’ views and satisfaction with their vascular access. The survey (VAQ) was administered by six trained research assistants to all in-center hemodialysis patients at the UHN-TGH who could speak English and were willing to partake in the survey. The pilot version of the VAQ was administered between May 1 and August 31, 2011 and took approximately 90 minutes per patient to complete. The final SF-VAQ was administered to a new sample of in-center hemodialysis patients (independent from the pilot testing group) on two occasions 2 weeks apart. It took approximately 10 minutes to complete the final SF-VAQ.

Questionnaire development

We undertook four key steps to develop our measurement tool: 1) item generation, 2) item selection, 3) item scaling and 4) psychometric evaluation.

1) In the item generation process, a comprehensive list of items was generated related to patients’ views and satisfaction with their vascular access by a team composed of two nephrologists, a pharmacist, a vascular access coordinator and a hemodialysis nurse. Over 150 items were proposed. They covered a majority of complications that could be associated with different interventions on each type of vascular access. These items were then pilot tested.

2) In the item selection process, the pool of items was analyzed based on predetermined and validated thresholds used in health measurement research (see Factor Analysis in Fig. 1). Factor analysis is a data reduction technique that identifies highly correlated items that can be grouped together in domains. Eigenvalues are used to select which factors (domains) to retain. After selection, the remaining 13 items were then reworded based on patient feedback during the pilot testing to improve comprehensibility, limit overlap and improve clarity. The items were then split into four domains for the final SF-VAQ: overall satisfaction (one item), physical symptoms (four items), social functioning (four items) and complications (four items) (see excerpts from questionnaire in Figure 2).

Fig. 1 - Fig. 2 -

3) In the item scaling process, the unipolar 5-point Likert scale used in the pilot testing phase was replaced with a bipolar 7-point Likert scale with a neutral middle in the final SF-VAQ. This resolved the pseudo-dichotomization of results seen in pilot testing where patients had trouble distinguishing between the “not at all” end of the scale when they actually felt “neutral.” In the 7-point scale used in the SF-VAQ, it is inferred that the “distance” between each successive Likert items is equidistant (20). The Likert scale, while ordinal, has sufficient symmetry (equal number of positive and negative statements) and equidistance to approximate an interval-level measurement (Fig. 2).

4) In psychometric evaluation of the finalized SF-VAQ, three metrics were used to evaluate its performance according to classical test theory: the test retest reliability, the internal consistency and the standard error of measurement. The test retest reliability is the proportion of all the variability that is due to changes within participants over time (20). The internal consistency is a measure of the degree to which the items correlate (20). The standard error of the measure provides an estimate of the precision of the final SF-VAQ.

Analysis

In the primary analysis, the mean scores for catheters, fistulas and grafts were summarized using descriptive statistics, and compared using an analysis of variance. The primary analysis was conducted using SPSS 22©.

For item selection following pilot testing of the VAQ, a factor analysis was performed in which factors were selected based on high eigenvalues (above 1.0) and clinical plausibility. For psychometric testing of the SF-VAQ, a repeated measures analysis of variance was conducted to obtain an estimate of the total variance and the variance attributable to item and time (i.e., the internal consistency and test–retest reliability) and the standard error of measurement. In classical test theory, any observed score includes the true score and some error variance. The items and time are both sources of error variance. Time is variable and as such a random factor, whereas the number of items does not vary and as such is a fixed factor; therefore, in the repeated measures analysis of variance the absolute error term was used in the calculation of test–retest reliability and the relative error term was used for the calculation of internal consistency. All psychometric analyses were performed using GENOVA©.

Results

One hundred and sixty four in-center hemodialysis patients at UHN-TGH participated in the pilot testing phase of this study and were administered the pool of 150 items. The final 13 item SF-VAQ was administered to a different group of 132 in-center hemodialysis patients on two occasions 1 week apart from February 1, 2012, to April 1, 2014.

Patient satisfaction with their vascular access

Of the 132 patients who completed the final SF-VAQ, 49 were using permanent access (35 fistulas and 14 grafts), and 83 were using catheters as their primary vascular access for dialysis. For question 3, where patients indicated their level of agreement with this statement: I am satisfied with my vascular access (where 1 is Strongly Disagree and 7 is Strongly Agree (Fig. 3)), the mean and median Likert values for all accesses were 5.98 and 7, respectively. This indicates that patients were quite satisfied with their vascular access overall. Patients using catheters, fistulas and grafts as their primary accesses had a mean Likert value of 5.92, 6.46 and 5.21, respectively. This indicates that patients were least satisfied with grafts and most satisfied with fistulas overall.

Fig. 3 -

Scores across the physical domain, where patients were asked to indicate the level of complications they had experienced with their vascular access scores were all below 4. This means that patients were overall satisfied with their access (low scores indicate satisfaction, 4 indicates neutrality and high scores indicate dissatisfaction in this domain). In the physical domain, which includes pain, bleeding, swelling and bruising, the item associated with the highest level of dissatisfaction overall was pain, with mean values of 1.49, 2.03 and 3.00 for catheters, fistulas and grafts, respectively (Tab. I). The mean sum scores for dissatisfaction across the four items in the physical domain was 5.4, 7.1 and 9.3 for catheters, fistulas and grafts, respectively, out of a possible 28 points (4 questions multiplied by 7 points of the Likert scale) (Tab. II).

TABLE I -

Results of the factor analysis in a scree plot.

Excerpts from SF-VAQ.

Overall satisfaction with access.

Mean and median Likert scores for questions 4-15 on the vascular access questionnaire by access type

Domain	Item (question number)	All accesses		Catheter		Fistula		Graft
		Mean	Median	Mean	Median	Mean	Median	Mean	Median
See Fig. 2 for excerpts from the questionnaire. In these domains, if a patient indicated 1-3, they agreed there was low or no problems for that item, if they indicated 4 they had no opinion, if they indicated 5-7 they agreed there was serious problems for that item.
Physical	Pain (4)	1.80	1.00	1.49	1.00	2.03	1.00	3.00	1.00
	Bleeding (5)	1.55	1.00	1.35	1.00	1.74	1.00	2.21	1.00
	Swelling (6)	1.49	1.00	1.33	1.00	1.66	1.00	2.07	1.00
	Bruising (7)	1.43	1.00	1.24	1.00	1.66	1.00	2.00	1.00
Social functioning	Daily activities (8)	2.18	1.00	2.11	1.00	2.43	1.00	2.00	1.00
	Appearance (9)	2.05	1.00	2.07	1.00	1.63	1.00	2.93	1.50
	Sleep (10)	1.98	1.00	2.13	1.00	1.57	1.00	2.14	1.00
	Bathing and showering (11)	2.61	1.00	3.27	1.00	1.60	1.00	1.29	1.00
Dialysis complications	Problem on dialysis (12)	1.95	1.00	1.88	1.00	2.00	1.00	2.29	1.00
	Access care (13)	1.54	1.00	1.66	1.00	1.40	1.00	1.14	1.00
	Hospitalization (14)	1.55	1.00	1.72	1.00	1.29	1.00	1.21	1.00
	Worry about access longevity (15)	3.12	2.00	3.13	2.00	3.11	1.00	3.07	1.50

TABLE II -

Mean and median sum scores within domains

		Sum of scores (by domain)
Domain	Metric	Overall	Catheter	Fistula	Graft
*Out of a possible 84 points from the 12 complication related questions in the questionnaire multiplied by 7 points of the Likert scale.
†Out of a possible 28 points from the 4 questions within a domain multiplied by 7 points of the Likert scale.
Overall (all three domains)*	Mean	23.3	23.4	22.1	25.4
	Median	20.5	20.0	18.0	22.5
	1st quartile	14.5	15.5	12.0	18.0
	3rd quartile	31.0	28.5	30.5	32.5
Physical†	Mean	6.3	5.4	7.1	9.3
	Median	4.0	4.0	4.0	8.0
	1st quartile	4.0	4.0	4.0	4.0
	3rd quartile	8.0	4.0	10.0	13.0
Social functioning†	Mean	8.8	9.6	7.2	8.4
	Median	8.0	8.0	4.0	8.0
	1st quartile	4.0	4.0	4.0	4.0
	3rd quartile	11.0	12.5	9.5	11.5
Dialysis complication†	Mean	8.2	8.4	7.8	7.7
	Median	7.5	8.0	7.0	6.5
	1st quartile	4.0	4.0	4.0	4.0
	3rd quartile	10.0	11.0	10.0	9.8

Similarly, scores across the social domain, where patients were asked to indicate the level of inconvenience to their social functioning because of their vascular access, were all below 4, meaning that patients were satisfied with their access. In the social functioning domain, which includes daily activities, appearance, sleep and bathing, the item associated with the highest level of dissatisfaction overall was bathing, with values of 3.27, 1.60 and 1.29 for catheters, fistulas and grafts, respectively (Tab. I). The mean sum scores for dissatisfaction across the four items in the social functioning domain were 9.6, 7.2 and 8.4 for catheters, fistulas and grafts, respectively, out of a possible 28 points (Tab. II).

In the dialysis complications domain, which includes problems on dialysis, vascular access care, hospitalization and concerns about vascular access longevity, the item associated with the highest level of dissatisfaction overall was concerns about access longevity, with values of 3.13, 3.11 and 3.07 for catheters, fistulas and grafts, respectively (Tab. I). The mean sum scores across the four items in the dialysis complication domain were 8.4, 7.8 and 7.7 for catheters, fistulas and grafts, respectively, out of a possible 28 points (Tab. II).

When the scores were summed across the physical, social functioning and dialysis complications domain, the mean sum scores were 23.4, 22.1 and 25.4, and median sum scores were 20.0, 18.0 and 22.5 for catheters, fistulas and grafts, respectively, out of a possible 84 points (12 questions multiplied by 7 points of the Likert scale). This indicates that patients reported the highest levels of dissatisfaction with grafts, followed by catheters and then fistulas having the lowest levels of dissatisfaction.

Psychometric evaluation of the SF-VAQ

The test–retest reliability of the SF-VAQ for two occasions was 0.92. According to classical test theory, this indicates that respondents were very consistent in their responses in a time period in which the trait was likely to have remained stable.

The internal consistency of the SF-VAQ for the first administration was high at 0.84. The internal consistency is above 0.9 within domains. Higher internal consistency within domains compared with overall suggests that the items within a domain more strongly correlate, which is a positive indicator that the questions within domains were appropriately selected.

The obtained standard error of 0.28 indicates that the error of measurement associated with any individual score is 28% of the standard deviation (21).

Discussion

We found that patients were overall satisfied with their vascular access—fistulas were the access type that patients were most satisfied with. The access type associated with the highest level of dissatisfaction was different in each domain: grafts were highest for physical complications while catheters were highest for social functioning and dialysis complications. This suggests that different access types have different weaknesses from a patient perspective.

In terms of the psychometric properties of the SF-VAQ, it was found to be precise, having high internal consistency and high test–retest reliability and it specifically ascertains hemodialysis patients’ views and satisfaction with their vascular access. This contrasts with generic quality of life measures such as EuroQOL and HUI3, which present with problems with sensitivity (22), particularly in the context of hemodialysis vascular access care program assessment. The EuroQOL and HUI3 both have the dual useful properties of being able to measure quality of life and the ability to assign a numerical value, a utility, to different health states for use in economic analysis (17). Both have been previously used to measure quality of life differences in modality choice for the treatment of end-stage kidney disease (13, 15, 23-24-25). A vascular access care program is anticipated to have an impact on patient quality of life; however, it may not be represented on the generic scale yet still have an impact on the utility score. Additionally, there are temporality problems associated with trying to accurately capture quality of life changes associated with vascular access-related complications such as infection and thrombosis; for example, if a patient has an infection at the fourth month on dialysis, this massive drop in their utility will not be captured by a measurement taken at the sixth month. Therefore, a generic measure such as the HUI3 is not an appropriate measurement tool in the context of vascular access care program evaluation.

The kidney disease-specific quality of life questionnaire and the Kidney Disease Health-Related Quality of Life (KDHRQOL) (26) are more disease-specific measures of quality of life that have more relevance to hemodialysis patients; the KDHRQOL has been previously validated in the hemodialysis population (12, 27, 28) and has nine disease-specific measures. However, the majority of the items are not relevant to hemodialysis patients in the context of vascular access care, and the length of the questionnaire places a significant burden on patients. Additionally, ascertaining and changing patient satisfaction with their vascular access rather than their quality of life may be more critical for ensuring patient adherence to their vascular access care program.

Validity

Given the rigorous nature of the item generation and selection undertaken in this study, the finalized SF-VAQ has high content validity. That is to say, the SF-VAQ items are representative of the dimensions of patient satisfaction with their vascular access. The higher the content validity of a measure, the broader the inferences that can be drawn from that measure under different conditions and situations. The overall study result indicating that fistulas generally performed better on the SF-VAQ than catheters provides some construct validation for the SF-VAQ. This is because it has been well demonstrated that fistulas are associated with lower morbidity than catheters, therefore presumably patients would be overall more satisfied with fistulas than catheters.

Limitations

This study has several limitations. It was conducted in English-speaking hemodialysis patients in one academic center in southern Ontario. As such the generalizability of these findings may be limited.

Future research

The SF-VAQ remains to be fully validated, especially for use in intervention evaluation. Construct validation is the optimal method of validating the SF-VAQ as it works well for hypothetical constructs like quality of life and patient satisfaction, especially in the absence of a “gold standard” measure (20, 29). The construct validation of the SF-VAQ is currently underway as part of a multicenter randomized clinical trial, the Hemodialysis Infection Prevention with Polysporin Ointment using Shower Technique in Satellite Centres (HIPPO-SAT) pilot study, comparing hemodialysis catheter care protocols (30). The validation of the SF-VAQ in the HIPPO-SAT pilot study for intervention evaluation is the critical last step in the development of this measurement tool of patient satisfaction with their vascular access.

Conclusions

Overall, according to scores reported on the SF-VAQ, patients were most satisfied with fistulas. After undergoing a thorough item selection and scaling process, the SF-VAQ is a short questionnaire that is easy to understand and administer with high content validity and robust psychometric properties. It contains items that are relevant to vascular access and is designed to detect real changes within patients and their respective vascular accesses over time. The validity testing of SF-VAQ to evaluate patient satisfaction with interventions in vascular access is underway as part of an ongoing clinical trial. The SF-VAQ will have utility in multiple research designs, from randomized control trials to quality assurance projects, to measure patients’ satisfaction with their vascular access.

• 1. Thomson PC.,Stirling CM.,Geddes CC.,Morris ST.,Mactier RA. Vascular access in haemodialysis patients: a modifiable risk factor for bacteraemia and death. QJM 2007; 100: 415-422
• 2. Ishani A.,Collins AJ.,Herzog CA.,Foley RN. Septicemia, access and cardiovascular disease in dialysis patients: the USRDS Wave 2 study. Kidney Int 2005; 68: 311-318
• 3. Engemann JJ.,Friedman JY.,Reed SD. Clinical outcomes and costs due to Staphylococcus aureus bacteremia among patients receiving long-term hemodialysis. Infect Control Hosp Epidemiol 2005; 26: 534-539
• 4. Marr KA.,Kong L.,Fowler VG.,Gopal A.,Sexton DJ.,Conlon PJ. Incidence and outcome of Staphylococcus aureus bacteremia in hemodialysis patients. Kidney Int 1998; 54: 1684-1689
• 5. Maraj S.,Jacobs LE.,Kung SC.,Raja R.,Krishnasamy P.,Maraj R. Epidemiology and outcome of infective endocarditis in hemodialysis patients. Am J Med Sci 2002; 324: 254-60
• 6. U.S. Renal Data System. USRDS 2009 Annual Data Report: Atlas of Chronic Kidney Disease and End-Stage Renal Disease in the United States Bethesda, MD National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases 2009
• 7. Ramanathan V.,Chiu EJ.,Thomas JT.,Khan A.,Dolson GM.,Darouiche RO. Healthcare costs associated with hemodialysis catheter-related infections: a single-center experience. Infect Control Hosp Epidemiol 2007; 28: 606-609
• 8. Mokrzycki MH.,Zhang M.,Cohen H.,Golestaneh L.,Laut JM.,Rosenberg SO. Tunnelled haemodialysis catheter bacteraemia: risk factors for bacteraemia recurrence, infectious complications and mortality. Nephrol Dial Transplant 2006; 21: 1024-1031
• 9. Tanriover B.,Carlton D.,Saddekni S. Bacteremia associated with tunneled dialysis catheters: comparison of two treatment strategies. Kidney Int 2000; 57: 2151-2155
• 10. U.S. Renal Data System USRDS 2010 Annual Data Report: Atlas of Chronic Kidney Disease and End-Stage Renal Disease in the United States. Costs of end-stage renal disease Bethesda, MD National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases 2010
• 11. Treatment of End-stage Organ Failure in Canada, 1999 to 2008; : -
• 12. Danquah FV.,Wasserman J.,Meininger J.,Bergstrom N. Quality of life measures for patients on hemodialysis: a review of psychometric properties. Nephrol Nurs J 2010; 37: 255-269;
• 13. Kimmel PL.,Cohen SD.,Weisbord SD. Quality of life in patients with end-stage renal disease treated with hemodialysis: survival is not enough! J Nephrol 2008; 21: S54-S58
• 14. Boyle MH.,Furlong W.,Feeny D.,Torrance GW.,Hatcher J. Reliability of the Health Utilities Index–Mark III used in the 1991 cycle 6 Canadian General Social Survey Health Questionnaire. Qual Life Res 1995; 4: 249-257
• 15. Lindsay RM.,Heidenheim PA.,Nesrallah G.,Garg AX.,Suri R;. Daily Hemodialysis Study Group London Health Sciences Centre. Minutes to recovery after a hemodialysis session: a simple health-related quality of life question that is reliable, valid, and sensitive to change. Clin J Am Soc Nephrol 2006; 1: 952-959
• 16. Mucsi I.,Molnar MZ.,Rethelyi J. Sleep disorders and illness intrusiveness in patients on chronic dialysis. Nephrol Dial Transplant 2004; 19: 1815-1822
• 17. Minshall ME. Cost-utility analysis: the nirvana to health economists and an enigma to decision-makers? Expert Rev Pharmacoecon Outcomes Res 2002; 2: 3-5
• 18. Quinn RR.,Lamping DL.,Lok CE. The Vascular Access Questionnaire: assessing patient-reported views of vascular access. J Vasc Access 2008; 9: 122-128
• 19. Kosa S Al Jaishi A Joarder B Tam P Moist L Lok C Hemodialysis Patient Views on their Catheter Care Protocols. Canadian Society of Nephrology Annual General Meeting St John’s, Newfoundland Canadian Society of Nephrology 2012
• 20. Norman G Streiner DL Health measurement scales: a practical guide to their development and use. New York, USA Oxford University Press 2008
• 21. Streiner DL.,Norman GR. “Precision” and “accuracy”: two terms that are neither. J Clin Epidemiol 2006; 59: 327-330
• 22. Guyatt GH.,Feeny DH.,Patrick DL. Measuring health-related quality of life. Ann Intern Med 1993; 118: 622-629
• 23. Wyld M.,Morton RL.,Hayen A.,Howard K.,Webster AC. A systematic review and meta-analysis of utility-based quality of life in chronic kidney disease treatments. PLoS Med 2012; 9: -
• 24. Mucsi I.,Kovacs AZ.,Molnar MZ.,Novak M. Co-morbidity and quality of life in chronic kidney disease patients. J Nephrol 2008; 21: S84-S91
• 25. Heidenheim AP.,Muirhead N.,Moist L.,Lindsay RM. Patient quality of life on quotidian hemodialysis. Am J Kidney Dis 2003; 42: 36-41
• 26. Diamant MJ.,Harwood L.,Movva S. A comparison of quality of life and travel-related factors between in-center and satellite-based hemodialysis patients. Clin J Am Soc Nephrol 2010; 5: 268-274
• 27. Laupacis A.,Wong C.,Churchill D. The use of generic and specific quality-of-life measures in hemodialysis patients treated with erythropoietin. Control Clin Trials 1991; 12: 168S-179S
• 28. Barotfi S.,Molnar MZ.,Almasi C. Validation of the Kidney Disease Quality of Life-Short Form questionnaire in kidney transplant patients. J Psychosom Res 2006; 60: 495-504
• 29. Lynn MR. Determination and quantification of content validity. Nurs Res 1986; 35: 382-385
• 30. Lawrence-Murphy JA.,Seiler S.,Hooton A.,Pember A.,St. Roch C. Shower and no-dressing technique for tunneled central venous catheters: an update one year later. CANNT J 2010; 20: 18-