The Validity and Reliability of the Urostomy Education Scale: A Methodological Study

© 2024 HMP Global. All Rights Reserved.

Any views and opinions expressed are those of the author(s) and/or participants and do not necessarily reflect the views, policy, or position of Wound Management & Prevention or HMP Global, their employees, and affiliates.

Urostomy is a surgical method performed by forming an opening in the abdominal wall through a short intestinal segment to either temporarily or permanently divert urine flow to exit the body when normal excretion of urine is obstructed.^1-3 It is most often performed in conjunction with radical cystectomy in cases of neurogenic bladder, urinary incontinence, trauma, radiation damage, or invasive bladder cancer. Bladder cancer is the 10th most common type of cancer worldwide and the fourth most common in Türkiye.⁴ The precise number of individuals living with a urostomy in Türkiye is not known, as statistics beyond institutional patient records are lacking.⁵ However, it is estimated that this number increases with the prevalence of bladder cancer.⁶

Urostomies cause loss of important body function, change in body image, and deterioration of sexual function, as well as physical, psychological, and social changes in lifestyle.^3,7-8 Therefore, individuals with urostomies should be taught coping strategies and trained in both pre- and post-urostomy surgery care.^9,10

Urostomy education should include not only general information about urostomies but also instructions regarding peristomal skin care, activities of daily living with a urostomy, diet, usage of urostomy supplies, urostomy bag changing skills, self-care management, and follow-up care with a wound, ostomy, and continence (WOC) nurse.^1,11 Application of skills related to self-care is a first and important step towards the acceptance of urostomy care independence after surgery. Basic skills include emptying, cleaning, and replacing the urostomy bag; observing the urostomy and peristomal skin; and inserting the new adapter/bag.^1,9,11

Individuals’ ability to fully perform the skills related to urostomy care is defined as stoma self-care,¹² which may be affected by factors such as age, sex, education, surgical technique, and physical/mental limitations.¹³ In a retrospective study of 65 patients who had undergone a urostomy, Tal et al reported that 52.3% of patients were able to care for their urostomies independently, and 49.2% of the patients felt that they were adequately prepared for stoma care after discharge.¹⁴ A study conducted by Person et al determined that 77.1% of individuals performed stoma care themselves, and 32.9% received help from their families.¹⁵ Concrete evidence shows that the ability to perform ostomy self-care is the most important variable in predicting positive adjustment to living with a urostomy.^16,17 A randomized, controlled study by Jehnsen et al evaluating the effect of preoperative training on postoperative self-efficacy determined that the preoperative training group had higher self-efficacy scores, although there was no significant difference between the groups.¹⁶ Similarly, Zganjar et al reported that structured preoperative ostomy training provided to patients scheduled for a urostomy increased patient compliance with living with a urostomy.¹⁷

As health professionals, nurses continually interact with patients to provide holistic care. WOC nurses aim to protect, develop, and maintain health through the planned training they provide to patients and their families. This training focuses on both skill acquisition and individuals’ ability to gain control over their own lives.¹⁸ Generally, urostomy self-care training takes place during the patient’s hospital stay after surgery. Because improved surgical techniques have shortened the average length of hospital stay after surgery, many patients are discharged without acquiring all of the knowledge and skills needed for urostomy care,¹⁹ which can delay urostomy management.

Post-discharge follow-up of patients with ostomies may be limited and educational needs insufficiently identified. Therefore, evidence suggests that nurses should follow a standardized training plan when teaching ostomy care. Established scales should be used to standardize and improve care and to monitor and evaluate care outcomes. These scales can also be used to maintain objective records. Because psychomotor skills include both cognitive and affective competencies, reliable measurement tools can provide objective data for the assessment of such skills.¹⁸

According to the literature, several instruments are available to define skin problems,^20-22 monitor compliance,^23,24 and quality of life,^25,26 and evaluate the personal care and education needs of individuals with urostomies.¹² Although studies have been conducted in individuals with urostomies in Türkiye, there are currently no established scales that allow WOC nurses to evaluate individuals’ urostomy care skills. Therefore, this methodological study aimed to examine the Turkish validity and reliability of the Urostomy Education Scale (UES) developed byKristensen et al.¹² In Türkiye, the UES can be used to provide individualized stoma care training and evidence-based care, as well as to conduct studies comparing the care skills of individuals with urostomies at the international level. The results of study presented here can contribute to the use of a standardized nursing approach when providing urostomy care training, thereby enhancing the quality of such training. In addition, it may contribute to systematic patient assessment for nurses who are responsible for promoting stoma self-care competence.

Aim of the study. This study was conducted to investigate the Turkish validity and reliability of the UES.

Study question. Is the UES a valid and reliable scale suitable for use in Turkish society?

Design of the study. This study was a methodological study.

Study population and sample. The study population consisted of 100 patients who underwent urostomy surgery at Istanbul University Cerrahpaşa Medical Faculty Hospital between May 2019 and April 2022. The sample consisted of patients who had undergone a urostomy, were in the 0- to 7-day postoperative period, were older than 18 years of age, had no physical or mental disabilities, had no urostomy complications, and who agreed to participate in the study. Because the UES is a 7-item scale and the recommended sample size for methodological research is 5 to 10 times greater than the number of items in the scale, a total of 66 patients were included in the study sample.²⁷

Data collection instruments. The data collection form consisted of 2 parts. The first part contained 11 questions developed by the researchers; these questions concerned the patients’ age, gender, marital status, educational status, occupation, income status, presence of chronic diseases, date of urostomy surgery, urostomy complication, and experiences of nausea and pain. The second part of the form contained the 7-item UES, developed by Kristensen et al.¹²

The UES was developed based on a literature review, and its 7 self-care urostomy bag system skills were determined by identifying the areas that are accepted as standard practice in urostomy care. These skills concern reaction to the stoma, removal of the stoma device, measurement of the stoma diameter, size adjustment of the urostomy diameter in a new stoma application, skin care, fitting of a new stoma device, and emptying procedure (emptying bag and attaching/detaching night bag). Each skill is scored with a 4-point rating scale according to the patient’s request for nurse assistance. Possible scores range from 0 to 3 points. A score of 0 points indicates that the patient is completely dependent on the nurse, 1 point indicates that the patient participates in the skill alongside nurse assistance, 2 points indicate that the nurse uses verbal guidance to help patients complete the skill, and 3 points indicate that the patient is able to complete the skill independently of the nurse. The total score that patients can receive from this training scale ranges from 0 to 21 points, with higher scores indicating higher skill acquisition (Appendix 1).¹²

Language equivalence of the scale. In the first stage of this study, the UES was translated into Turkish for linguistic adaptation by 3 independent bilingual (English and Turkish) linguistic experts. All translations were converted into a single text by the authors. Prior to back translation, the experts who are certified WOC nurses and nursing faculty members are consulted. Then, the scale was back-translated by 3 independent linguistic experts.

Content and face validity of the scale. The content validity of the scale was assessed by 3 faculty members certified in ostomy and wound care and 2 WOC-certified nurses. Each was asked to rate the items on a 4-point Likert scale as follows: 4, very relevant; 3, relevant; 2 partially relevant; and 1, not relevant. They were also asked to note any suggestions for changes. The final version of the scale was created after consulting expert opinions of certified 2 WOC nurses and 3 nursing faculty members. To determine the face validity of the scale items, a pilot study was conducted in which 2 competent and 2 expert nurses evaluated 5 individuals with urostomies. The nurses then rated their level of comprehension of the scale (3, very clear; 2, slightly clear; and 1, not clear).^27,28 No further changes were made following the pilot study, and all items on the scale were considered comprehensible.

Data collection. The study data were collected between May 2019 and April 2022. One competent and 1 expert nurse were enrolled in the study. Five of the 13 nurses working in the urology ward where the research was conducted worked day shifts and were considered competent (defined as having 2 or more years of experience), 1 of whom was selected for and agreed to participate in the study. Because only 1 nurse in the hospital where the research was conducted held a wound care certificate, she participated as an expert nurse.

After obtaining institutional approvals, the nurses who were to assess the self-care skills of individuals with urostomies and who agreed to participate in the study were given 45 minutes of theoretical education and 2 hours of practical education designed to explain the purpose of the UES, its application, and the meaning of the scores assigned by the expert. The individuals with urostomies included in the study were informed about the purpose of the study, the forms to be completed, and their ability to perform urostomy self-care during their hospitalization. The patients were evaluated 3 times by 2 nurses on postoperative days 1 or 2, 3 or 5, and 6 or 7. Each patient evaluation lasted approximately 30 minutes. Patients’ urostomy self-care skills were evaluated separately by the nurses as soon as the urostomy care was performed. Before each evaluation, patients were asked to rate their levels of pain and nausea on a scale of 1 to 10. Subsequently, upon resolution of any signs and symptoms of nausea or pain, urostomy care was provided.

Ethical considerations. Ethical approval was obtained from Koç University Social Sciences Ethics Committee (2019.225.IRB3.118), and institutional permission was taken from the hospital where the research was conducted. Permission for the adaption of the scale was obtained from the scale’s authors via email. Written and verbal consent were obtained from the patients and nurses who agreed to participate in the study, and the purpose of the study was explained to participants before each evaluation. Individuals with urostomies who agreed to participate in the study were not given any incentive for participation.

Data analysis. The SPSS (Statistical Package for the Social Sciences) version 25.0 (IBM Corp) program was used for statistical analysis of the research findings. Descriptive, graphical, and statistical methods were used to examine whether the scores obtained from each variable were normally distributed. The Shapiro‒Wilk test was used to test the normality of the scores obtained from a continuous variable by statistical methods. The content validity of the scale was calculated using the Content Validity Index (CVI). In addition to descriptive statistical methods (number, percentage, mean, median, standard deviation, etc), the Mann‒Whitney U test was used to compare quantitative data of the 2 groups, and more than 2 groups were compared using the Kruskal‒Wallis test. In the Kruskal‒Wallis test, a Bonferroni correction was applied to determine which groups caused the difference. The correlation level between the 2 continuous variables was examined using the Spearman correlation test. While the Friedman test was used to test the difference in multiple repetitive measurements, the Wilcoxon test was used to test the difference in 2 groups of repetitive measurements. The internal consistency of the scale was calculated with Cronbach’s alpha coefficient, and the degree of reproducibility (agreement between scores) of the scale scores was calculated with the intraclass correlation coefficient (ICC). The results were analyzed at a 95% CI and significance level of P < .05.

Patient characteristics. A total of 66 patients were included in the study. Patient skill evaluations were carried out 3 times by competent and expert nurses. The mean age of the patients was 53.8 (standard deviation [SD]: 16.1) years, with 58% being male, 74% being married, 52% being primary school graduates, 52% being unemployed, 67% having expenses exceeding their income, and 32% having been diagnosed with a chronic disease. The mean postoperative pain level of the patients was 3.2 (SD: 1.3) (Table 1).

Validity analysis. According to expert opinion, the CVI of the Turkish version of the scale was 0.81, indicating excellent content validity.²⁸

Reliability of the scale

Internal consistency of the scale. Cronbach’s alpha (α) coefficients were calculated to determine the scale’s internal consistency in the mean of 3 evaluations made by the competent and expert nurse for UES items. As a result of these calculations, it was determined that α = 0.66 to 0.95 in the competent nurse evaluation and α = 0.68 to 0.96 in the expert nurse evaluation (Table 2).

Mean UES scores. On postoperative days 1 or 2, 3 or 5, and 6 or 7, the competent nurse’s overall mean scores were 0.58 ± 1.58, 1.80 ±  2.41, and 5.14 ±  6.70, respectively. The expert nurse’s overall mean ratings were 0.59 ± 1.59, 2.08 ± 3.39, and 5.35 ± 6.97, respectively. The difference between the mean skill evaluation scores of the 2 nurses was found to be statistically significant (competent nurse, χ2 = 76.10; specialist nurse, χ2 = 82.54; P < .001). This difference was statistically significant in the subgroup analyses for both the competent and expert nurse evaluations across all measurement times (P < .001). In both time evaluations, the self-care skill levels of the patients increased from the first evaluation to the last. The evaluation results of each patient urostomy self-care skill are shown in detail in Table 3.

Interrater reliability of the study. There was no statistically significant difference between the evaluations made by the competent and expert nurses for each skill on postoperative days 1 or 2, 3 or 5, and 6 or 7 (P  > .05). However, the skills with the greatest difference between the 2 evaluators were the second skill (removal of the stoma device) on the first/second day (P = .083), the third skill (measurement of the stoma diameter) on the third/fifth day, the fourth skill (adjustment of the size of the urostomy diameter for a new stoma device) on the third/fifth day, the third skill (measurement of the stoma diameter) on the sixth/seventh day, and the seventh skill (discharge procedure) (P = .180) and first skill (reaction to the stoma) in the average of the 3 evaluations (P = .084; Table 4).

When the total mean scores of the 7 skills were examined, the difference between the patient urostomy self-care evaluation scores of the competent and expert nurses was  -0.02 (95% CI: -0.16-0.13; P = .903) on days 1 or 2, -0.27 (95% CI: -0.78-0.24; P  = .422) on days 3 or 5, -0.21 (95% CI: -0.64-0.22; P = .564) on days 6 or 7, and -0.17 (95% CI: -0.49-0.15; P = .955) in the average of the 3 periods; no statistically significant difference was found between the 2 evaluators (P > .05; Table 5).

Intraclass correlation coefficient. Agreement between the evaluation scores for UES items from the competent and expert nurses was examined by ICC. According to the ICC results, agreement between the evaluations made by the 2 evaluators for each skill was found to be sufficient (ICC range: 0.6-1) and statistically significant (P < .001; Table 4 and Table 5).

Valid and reliable scales used by stoma and wound care nurses to evaluate the self-care ability of individuals with a urostomy are not available in Türkiye. Therefore, this study was carried out to assess the reliability and validity of the UES in Turkish society. The study’s findings showed that the Turkish version of the UES is a viable and reliable tool that can be used to evaluate patient capacity for urostomy self-care.

The language validity in the relevant culture should be established prior to scales being utilized in multiple cultures. The CVI index was computed in this investigation using the Davis calculation method. The computation indicates that a CVI of 0.80 or above is normal. This demonstrates that there is agreement among the experts, who are certified WOC nurses and faculty members, and that the scale’s scope is adequate to assess urostomy patients’ self-care abilities.²⁸ In the original study of the scale, the CVI was evaluated using the Delphi method. The researchers reached a consensus of 80%, determining that the scale was exhaustive and exclusive.¹²

Reliability is the most basic feature that shows the extent to which a measurement tool accurately measures its intended feature and confirms continuity of the scale.^28,29 As the term implies, reliability refers to maintaining the same results when independent measurements are taken of the same item, following the same procedures, and using the same criteria to achieve the same results.³⁰ The scientific value of a scale with low reliability is also considered low. Multiple methods have been developed to test the reliability of a measurement tool. These are grouped under 3 headings: internal consistency, interobserver reliability, and invariance over time.^29,30 In this study, the reliability of the UES was examined using internal consistency and interobserver reliability tests.

One of the most commonly used methods to examine the validity of a measurement tool is the evaluation of the internal consistency of the scale. Cronbach’s alpha value for internal consistency is interpreted as “scale reliability” between 0.60 and 0.79 and “scale high reliability” between 0.80 and 1.00.²⁸ In this study, according to the evaluations made by the competent nurse (α = 0.66-0.95) and the expert nurse (α = 0.68-0.96), the internal consistency of the scale was determined to be acceptable and reliable.²⁸ The original Cronbach’s alpha scale and ICC management were not used. Instead, reliability was tested using the Bland-Altman Plot method, and the scale was found to be reliable.³¹

Interobserver reliability was measured with a single form used by 2 observers and examining the correlation between them. Interobserver reliability is a commonly used in observational data collection or in studies with more than 1 evaluator. Interobserver agreement is not associated with the measurement tools used and varies depending on the observers.³⁰ In this study, the interrater reliability of the UES of nurses with different levels of experience was analyzed. Based on this study, no differences were found between competent and expert nurses’ assessments on different days and for each skill. Similar to this study, no difference was found between nurse evaluators with different levels of experience in the original study.³¹

ICC is used in cases where there are 2 or more evaluators to measure reliability. The ICC value may be between 0 and 1; a value of 0 indicates that there is no agreement between the raters, while a value close to 1 indicates perfect reliability between raters.^30,32 In this study, there was sufficient agreement between the evaluations made by both evaluators for each skill (ICC range: 0.6-1).

It is important that the self-care skills of individuals with urostomies be monitored by WOC nurses and evaluated using a scale. Similar to the original study,³¹ this study found that self-care skills of individuals with a urostomy were low in the first day’s assessment but improved during the last days.

Use of the UES is limited to individuals with a urostomy and does not include individuals with stomas. A strength of the present study is that measurements were made at 3 different times for each urostomy individual. However, there were only 2 evaluators for the UES and the study was conducted with only 66 patients, which present limitations.

In conclusion, the original form of the 7-item UES translated into Turkish was found to be suitable for use by Turkish-speaking WOC nurses and has acceptable psychometric properties. Therefore, the scale can be used to assess the self-care skills of urostomy patients, especially by WOC nurses. This study may serve as a guide to achieve better patient-centered care and improved patient outcomes. Future studies in which the interrater reliability test is repeated with more than 2 evaluators are recommended.

Individuals with urostomies should be provided with structured education on the care of their stomas to improve their self-care skills. To maintain quality of care, standardized supportive care plans that allow for the identification of key areas of maintenance intervention must be in place. The UES may be used as a standard assessment tool to monitor the self-care skills of individuals with a urostomy. During the postoperative period, it is vital that individuals with urostomies acquire necessary self-care skills. The scale can be used by WOC nurses to determine, monitor, and improve the self-care skill levels of individuals with a urostomy. Thus, patient-centered care can be provided to individuals with a urostomy, thereby increasing self-care skills. Moreover, the Turkish version of the UES allows for the comparison of self-care abilities among individuals with urostomies, contributing not only to the establishment of a common language but also supporting further worldwide comparative studies.

Acknowledgments: The authors would like to thank all patient and nurse participants involved in this study.

Affiliations: ¹Istanbul University, Faculty of Nursing, Istanbul, Türkiye; ²Koç University School of Nursing, Istanbul, Türkiye; ³Istanbul University-Cerrahpaşa, Cerrahpaşa Medical Faculty Hospital Istanbul, Türkiye.

ORCID: Yiğitoğlu, 0000-0001-6204-1543; Karaçay, 0000-0002-5627-2836; Karadede, 0000-0002-3845-7423

Disclosures: The authors have no funding or conflicts of interest to disclose.

Correspondence: Eylem Toğluk Yiğitoğlu, PhD, Istanbul University, Faculty of Nursing,  Istanbul University Beyazit Campus, 34116 Fatih/İstanbul, Türkiye; eylem.togluk@istanbul.edu.tr

1. Berti-Hearn L, Elliott B. Urostomy care: a guide for home care clinicians. Home Healthc Now. 2019;37(5):248-255. doi:10.1097/NHH.0000000000000792
2. Igel DA, Chestnut CJ, Lee EK. Urinary diversion and reconstruction following radical cystectomy for bladder cancer: a narrative review. AME Med J. 2021;6:4. doi:10.21037/amj-20-76
3. Villa G, Manara DF, Brancato T, et al. Life with a urostomy: a phenomenological study. Appl Nurs Res. 2018;39:46-52. doi:10.1016/j.apnr.2017.10.005
4. GLOBOCAN 2020 Turkey Fact Sheets. International Agency for Research on Cancer (IARC), World Health Organization (WHO). Accessed January 18, 2024. https://gco.iarc.who.int/media/globocan/factsheets/populations/792-turkiye-fact-sheet.pdf
5. Yılmaz G, Harputlu D, Pala MI, et al. A cross sectional evaluation of patients with ostomy in Turkey. Turk J Colorectal Dis. 2021;31:136-142. doi:10.4274/tjcd.galenos.2020.2020-10-10
6. Cancer Statistics in Turkey 2018. T.C. Ministry of Health General Directorate of Public Health Department of Cancer. Accessed Jan 18, 2024. https://hsgm.saglik.gov.tr/depo/birimler/kanser-db/Dokumanlar/Istatistikler/Kanser_Rapor_2018.pdf  
7. Kandemir D, Oskay U. Sexual problems of patients with urostomy: a qualitative study. Sex Disabil. 2017;35:331-340. doi:10.1007/s11195-017-9494-8
8. Yi E, Yoo YS, Lee S, Park H. The experiences of illness in Korean bladder cancer patients with radical cystectomy. Cancer Nurs. 2022;45(2):132-140. doi:10.1097/NCC.0000000000000965
9. Wound, Ostomy and Continence Nurses Society; Guideline Development Task Force. WOCN Society clinical guideline: management of the adult patient with a fecal or urinary ostomy-an executive summary. J Wound Ostomy Continence Nurs. 2018;45(1):50-58. doi:10.1097/WON.0000000000000396
10. Regulation on making changes in the nursing regulation. Article in Turkish. Official Newspaper. April 19, 2011. Accessed November 19, 2022. https://www.saglik.gov.tr/TR,10526/nurse-yonetmeliginde-degisiklik-yapilmasina-dair-yonetmelik.html
11. Harputlu D. The care of patient with urostomy. Article in Turkish. Bull Urooncol. 2015;14:8-12. doi:10.4274/uob.228
12. Kristensen SA, Laustsen S, Kiesbye B, Jensen BT. The Urostomy Education Scale: a reliable and valid tool to evaluate urostomy self-care skills among cystectomy patients. J Wound Ostomy Continence Nurs. 2013;40(6):611-617. doi:10.1097/01.WON.0000436778.39349.12
13. Eraydın C, Sunal N. Self-care ability and affecting factors in patients with stoma. Article in Turkish. Akd Med J. 2021;7(1):57-64. doi:10.17954/amj.2021.2677
14. Tal R, Cohen MM, Yossepowitch O, et al. An ileal conduit--who takes care of the stoma? J Urol. 2012;187(5):1707-1712. doi:10.1016/j.juro.2011.12.064
15. Person B, Ifargan R, Lachter J, Duek SD, Kluger Y, Assalia A. The impact of preoperative stoma site marking on the incidence of complications, quality of life, and patient’s independence. Dis Colon Rectum. 2012;55(7):783-787. doi:10.1097/DCR.0b013e31825763f0
16.  Jensen BT, Kiesbye B, Soendergaard I, Jensen JB, Kristensen SA. Efficacy of preoperative uro-stoma education on self-efficacy after radical cystectomy; secondary outcome of a prospective randomized controlled trial. Eur J Oncol Nurs. 2017;28:41-46. doi:10.1016/j.ejon.2017.03.001
17. Zganjar A, Glavin K, Mann K, et al. Intensive preoperative ostomy education for the radical cystectomy patient. Urol Oncol. 2022;40(11):481-486. doi:10.1016/j.urolonc.2021.04.025
18.  İncesu O, Ulupınar S. Developing an assessment tool for stoma care skill: rubric. Ordu University J Nurs Stud. 2022;5(2):149-157. doi:10.38108/ouhcd.915456
19. Jensen BT, Lauridsen SV, Jensen JB. Optimal delivery of follow-up care after radical cystectomy for bladder cancer. Res Rep Urol. 2020;12:471-486. doi:10.2147/RRU.S270240
20.  Beitz J, Gerlach M, Ginsburg P, et al. Content validation of a standardized algorithm for ostomy care. Ostomy Wound Manage. 2010;56(10):22-38.
21. Ay A, Bulut H. Assessing the validity and reliability of the peristomal skin lesion assessment instrument adapted for use in Turkey. Ostomy Wound Manage. 2015;61(8):26-34.
22. Harputlu D, Özsoy SA. A tool to diagnose peristomal skin compliance: Ostomy Skin Tool. Florence Nightingale J Nurs. 2016;24(1):38-48.
23. Simmons KL, Smith JA, Maekawa A. Development and psychometric evaluation of the Ostomy Adjustment Inventory-23. J Wound Ostomy Continence Nurs. 2009;36(1):69-76. doi:10.1097/WON.0b013e3181919b7d
24. Karadağ A, Baykara ZG, Korkut H, Çelik B. Adaptation of the ostomy adjustment inventory into Turkish language. Article in Turkish. Turk J Surg. 2011;27(4):206-211. doi:10.5097/1300-0705.UCD.1321-11.01
25. Grant M, Ferrell B, Dean G, Uman G, Chu D, Krouse R. Revision and psychometric testing of the City of Hope Quality of Life-Ostomy Questionnaire. Qual Life Res. 2004;13(8):1445-1457. doi:10.1023/B:QURE.0000040784.65830.9f
26. Erol F, Vural F. Validity and reliability of The City of Hope Quality of Life Ostomy-Scale for the Turkish patients with ostomy. Hemar-G. 2012;14(3):1-14. http://hemarge.org.tr/ckfinder/userfiles/files/05_makale_1.pdf
27. Tavşancıl E. Reliability. In: Bunny E, ed. Measuring attitudes and data analysis with SPSS. 5th ed. Nobel Publications; 2019:16-33.
28. Polit DF, Beck CT. Nursing Research: Generating and Evaluating Evidence for Nursing Practice. 10th ed. Wolters Kluwer Health; 2017.
29. Karaçam Z. Adaptation of measurement tools to Turkish. Article in Turkish. J Midwifery and Health Sci. 2019;2(1):28-37.
30. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012
31. Kristensen SA, Jensen BT. Testing inter-rater reliability of the Urostomy Education Scale. Eur J Oncol Nurs. 2016;20:17-23. doi:10.1016/j.ejon.2015.06.004
32. Liljequist D, Elfving B, Skavberg Roaldsen K. Intraclass correlation - a discussion and demonstration of basic features. PLoS One. 2019;14(7):e0219854. doi:10.1371/journal.pone.0219854