Producing Precise Outcomes in Randomized, Controlled Trials and Clinical Studies

  Abstract: Evidence-based medicine (EBM) uses the current best evidence to inform decisions about care of individual patients, healthcare procedures, and technologies. The “gold standard” for optimal evidence in the Cochrane system is Level I randomized, controlled trials (RCTs) and meta-analyses of several RCTs. In order to achieve this level of evidence, one of the most important measures is the use of outcomes/endpoints. This article will provide, in short form, recommendations on how to achieve rigorous endpoints or outcomes in studies on wound management. Consistency in measuring endpoints/outcomes improves quality of care. To achieve such consistency it is important to 1) use predefined and robust outcomes; 2) adapt outcomes to the intervention under investigation; and 3) use the best evidence available. Also, it is emphasized that the use of complete wound closure or healing as an outcome measure is not always possible or suitable. Remaining patient-focused clarifies which other endpoints are relevant. Finally, “basic care” must be clearly defined and standardized when used as a comparative intervention in a RCT. In conclusion, the use of correct, clinically relevant outcomes or endpoints is of vital importance when establishing optimal evidence in wound healing and care.

Introduction

  Nonhealing wounds are a significant problem for healthcare systems all over the world. In the industrialized world, almost 1%–1.5% of the population has a problem wound at any one time and accounts for 2%–4% of the healthcare budget—a figure that is likely to rise with an increasing elderly and diabetic populations.^1–3   For these reasons, there is an urgent need to review wound strategies and treatments in order to reduce the burden of care in an efficient and cost-effective way. A primary question is which of the available interventions, technologies, and dressing materials achieve the best outcomes for the best value? Ongoing controversy surrounds the value of various approaches to wound management and care. Evidence-based clinical practice is difficult to achieve due to confusion about the value of the various approaches to wound management. To address this, the European Wound Management Association (EWMA) set up a Patient Outcome Group.⁴ The first document identifies criteria for producing precise outcomes in both RCTs and clinical studies, and describes how to ensure that studies are consistent and reproducible.⁵ This review will highlight the principles in the EWMA document.   Systematic reviews have indicated that there are substantial deficiencies in the quality of clinical research (eg, www.cochrane.org, www.nice.org.uk). Wound management has a paucity of high quality evidence, as the studies are often based on inadequate sample size, short follow up, nonrandom allocation to treatment arms, nonblinded assessment of outcomes, and poor description of control and concurrent intervention(s).   Quality of evidence in wound management is interesting from different perspectives:     1. From the clinical perspective the question is which interventions, technologies, and dressing materials are the best from the point of view of a single patient or group of patients, where the primary focus is healing and the absence of complications.     2. From the policy maker and healthcare system perspectives two issues arise: A) Whether or not a particular product or intervention is safe and effective when used as indicated—this is a question of regulatory approval, and B) whether or not the product or intervention represents a cost-effective use of funds. Too few high quality clinical or economic studies in wound care have resulted in challenges to the reimbursement of modern dressings in favour of supposedly better value traditional products.     3. From the industry perspective (medical device industry) the challenge is that the standard of care and evidence requirements for reimbursement are different in each country and that the large investments related to RCTs are rarely justified by the pace of innovation and size of markets of most wound care products.   The aim of the background article⁵ was to provide recommendations on how to achieve rigorous endpoints/outcomes in studies on wound management, and to describe an approach that will enable the design of RCTs and clinical studies to be both consistent and reproducible in order to reach a higher quality of evidence in wound management. These principles will be summarized with the goal of improving the consistency and quality of wound care outcomes measurement, documentation, and management around the world.

Methods

  Materials. The background information, evaluations, and recommendations of the document are based on a collection of a large amount of available guidance for evidence collection, and an analysis of recent RCTs and comparative studies. To achieve an updated status on how endpoints/outcome parameters are used, defined, and evaluated, the authors performed a literature search of RCTs on the Cochrane Database, MEDLINE, and Embase from 2003–2009 studying chronic or problem wounds or ulcers, with the objective of examining and registering the use of endpoints, the quality of endpoint definitions, and the robustness of methodologies. Randomized controlled or nonrandomized clinical or preclinical studies qualified for inclusion in the analysis if they reported a measured endpoint or outcome of any aspect of wound management. Reviews, opinion papers, and case reports were excluded.   Definition of endpoints/outcomes. Study endpoints or outcomes are the keystone in the evidence discussion. To clarify the basic premises of the discussion, the terminology needs to be clarified. An endpoint/outcome parameter is defined as the objective of an evaluation or study.^6–8 The objectives should include: 1) a precise statement of the degree of benefit expected from the intervention, and its duration; 2) clear statements on the time frame of the study (especially in relation to how quickly the benefits might start); and 3) a definition of the patients for whom the benefit is sought.   In the past, the most commonly used outcome in clinical trials was visible reduction in wound size, particularly restoration of intact skin (complete healing). The development of tests and techniques to improve tissue sampling and analysis, imaging technology, and scientific progress in cellular and molecular biology has enabled the development of more objective wound outcome parameters, such as transepidermal water loss (TEWL) for re-epithelization,⁹ and surrogate outcome parameters that relate to both the wound condition and the treatment intervention being assessed. Examples of objective wound condition outcome measures include, exudation rate,¹⁰ pain,¹¹ granulation rate,¹² resolution of necrosis,¹² or infection.¹³ Another objective measurement of a treatment intervention is the operational definition of moist wound healing in terms of the dressing moisture/vapor transmission rate.¹⁴   A surrogate endpoint/outcome parameter is defined as a physical sign or a laboratory measurement that can be used as a substitute for a clinically meaningful endpoint, effectively directly measuring how a patient feels, functions, or survives.¹⁵ The challenge in nonhealing wounds is that endpoints/outcome parameters, such as complete wound closure, are difficult to achieve and maintain. If the only gold standard were total wound closure, efficacious therapies capable of hastening partial wound closure or preparing wounds for successful grafting, may never be considered. Alternative clinical or surrogate endpoints are therefore needed.   Methods. To establish a basis for the recommendations and statements related to study designs and use of outcome measures in wound management, data collection issues, concepts, and terms were discussed by the 3 authors, authorized by EWMA and used in the document.   The author panel recommendations addressed: 1) study design to optimize wound care intervention or product safety and efficacy; 2) industry requirements to optimise product development, registration, and reimbursement; and 3) operational definitions of “standard care” including management of underlying disease.   From the clinical research perspective, there is a need to be aware of the strengths and limitations of different study designs if they are to effectively evaluate which health-care practices are worth considering for different patients in different health contexts. Key issues are, for example, use of a study protocol, problems related to heterogeneity of the study population, and underlying conditions. From an industry perspective, external evidence needs are set by the requirements of national regulatory and reimbursement authorities, and other payers. When developing a new product, there are also internal needs for evidence, which mirror the phases of the development process. When focusing on payment or reimbursement for a new product, the key issue will often be budgetary impact and/or cost-effectiveness, rather than healing.   The document also discusses the importance of a generally accepted definition of “standard care” in connection with data collection in wound management. “Standard care” refers to generally accepted wound care procedures and the management of underlying disease, outside of the investigational product/device or drug that will be used in the clinical trial/evaluation. It is essential that the standard care procedures/regimens used are consistent, as this will minimize variability and enable assessment of the treatment effect.

Results

  What types of outcomes and endpoint measures have been used in wound healing studies? Studies published before 2003 are described by Matousek et al7 and 2003–2009 studies by the EWMA Patient Outcome Group.⁵ In our literature search, abstracts from RCTs and comparative studies were evaluated, identifying 176 articles that qualified for analysis. Many studies measured multiple endpoints—in total, this analysis generated a list of 313 different endpoints. In Table 1 the most often found endpoints are mentioned.   The findings of the analysis were used as a basis for discussing and suggesting procedures for the successful measurement of each of the types of endpoint categories defined.   In general, it was found that a substantial number of endpoints (45%) were either not predefined or insufficiently defined. As part of the analysis, the degree of robustness of the measurement techniques used in studies* and the degree of reproducibility‡ were evaluated. In 70% of cases, a standardized or clearly defined measurement technique was used to examine the endpoint (eg, computerized planimetry or a standardized evaluation method). These were defined as meeting the criteria for an acceptable degree of robustness. However, 76% of these did not meet the criteria for reproducibility.   The analysis and discussion resulted in a number of recommendations (“statements”) on use of endpoints in RCTs and comparative studies on nonhealing wounds (Table 2).   What types of biases could influence outcome/endpoint evaluation of studies? When evaluating interventions in wound management it is a challenge to avoid bias. Designing studies with the aim of obtaining sufficient information regarding outcomes is particularly hazardous. The document, therefore, discusses and provides recommendations regarding some of the considerations that should be made when designing studies on interventions for the treatment of wounds.   Table 3 outlines some of the potential sources of bias that can be introduced into study design, unless great care is taken at the planning stage. In many cases, the issues are very similar regardless of intervention and are not specific to wound management studies; however, the key points related to the wound community are discussed below.   There is always the potential for bias to be introduced in studies, but the RCT design puts the emphasis on reducing bias as much as possible. Table 3 highlights the particular difficulties that may apply to wound management studies. There is always a debate about the nature of the design of the study, dependent on the audience for whom the data are prepared: for example, regulatory authorities require the purest form of an RCT based on a restricted population in order to reduce the heterogeneity of the population, to ensure that the study has sufficient internal validity to demonstrate efficacy. However, this restrictive approach to study design will not allow for the generalization of the findings to those patients who routinely present at the authors’ clinics, where a comparative effectiveness study with the emphasis on whether or not the treatment works pragmatically in routine practice, may be more appropriate. There are certain situations where the outcomes of an RCT may be very predictable, for example, using healing as an outcome for certain dressing trials: this contradicts a basic premise for conducting an RCT which states that the researchers should be in a state of equipoise (ie, uncertain about which intervention works best). In such circumstances, a comparative cohort study may be more appropriate, as the resources used to achieve similar outcomes is a more important question to investigate.   A further level of bias may be introduced if interventions are not used appropriately, in line with the manufacturer’s instructions, or as required by the wound condition. This is particularly difficult when a purist approach to RCT design requires that the same intervention be used throughout the study period, which would directly contradict the clinical need to adapt the treatment to the condition of the wound. There is a real tension between the requirement to stick to a purist approach and being pragmatic about the ways in which treatments are used in routine practice.   Recommendations for use of outcomes/endpoint. The analysis presented in a previous section resulted in a number of rigorous recommendations for outcomes/endpoints in studies on wound management RCTs and comparative studies on nonhealing wounds (Table 4). * Defined as a level of information about the measurement technique, such that others could replicate the data collection in an identical way. ‡ Defined as the inclusion of a verifiable source of data, eg, photos to secure the possibility of validation from an external source by reproducing the study, or the involvement of an external validation source as part of the study design.

Discussion

  The main challenge for establishment of a basis for the recommendations and statements related to study designs and use of outcome measures in wound management has been to control the heterogeneity of individual patients, concurrent disease states, and confounding factors, as well as variations in the type, site, and condition of wounds and differences in healthcare organizations. Enrolling more subjects into a study cannot solve these problems.   Currently, the majority of wound management studies recruit patients with one wound etiology. However, the development of more targeted strategies specific to different phases of treatment (eg, debridement) means that the condition of the wound (eg, exudate rate, pain, and necrosis) may be a better inclusion criterion.   When evaluating interventions in wound management it is also a challenge to avoid performance bias. Designing studies with the aim of obtaining sufficient information regarding outcomes is particularly hazardous. The document therefore discusses and provides recommendations regarding some of the considerations that should be made when designing studies on interventions for the treatment of wounds.   The definition of nonhealing wounds is still discussed. The authors suggest that the definition “chronic” be replaced with the definition “nonhealing,” as this better reflects the clinical problems experienced by such patients. It is also suggested that multicentre trials should imply great efforts to enrol sufficient numbers at each site and a high degree of protocol standardization. Another issue brought forward is the selection of an appropriate study period. Recommendations from various institutions vary from 2 weeks to 12 months. The authors recommend that the type(s) of ulcer and relevant natural outcome are considered when selecting the study duration.   Some common methodological errors in wound dressing studies and a checklist for objectives and endpoints/outcomes in clinical trials are included in this summary (Tables 2 and 3).‡ ‡ Tables 15 and 16 in the full document.

Conclusion

  This article has focused on the role of evidence in wound healing, especially the problems achieving the highest level of evidence. One of the most important measures in the process of producing evidence is the use of outcomes/endpoints. These measures have been described along with the importance of choosing rigorous and robust outcomes/endpoints in order to design both consistent and reproducible RCTs and clinical studies in order to reach a higher quality of evidence in wound management.   The key messages of the document can be summarized as follows:     1. A substantial number of publications exist, but the quality of evidence needs to rise.     2. Consistency in measuring endpoints/outcomes improves quality. To reach consistency it is important to:       • Use predefined and robust outcomes       • Adapt outcomes to the intervention under investigation       • Use the best evidence available.     3. Using intact skin/healing as an outcome measure is not always possible/suitable – using the patient focus clarifies which other endpoints are relevant.     4. The term “chronic” wound should be replaced with “nonhealing” wound.     5. Basic care must be standardized.     6. The Patient Outcome Group does not reject RCTs in wound management, but advocates that clinical trials using alternative endpoints to healing are evaluated as being equally suitable for the evaluation of various wound interventions. * Defined as a level of information about the measurement technique, such that others could replicate the data collection in an identical way. ‡ Defined as the inclusion of a verifiable source of data, eg, photos to secure the possibility of validation from an external source by reproducing the study, or the involvement of an external validation source as part of the study design.

References

1. Dale J. Chronic ulcers of the leg: a study of the prevalence in a Scottish community. Health Bulletin. 1983;41(6):310–314. 2. Liu PY, Eriksson E, Mustoe TA. Wound healing: practical aspects. In: Russels RC, ed. PSEF Instruction Courses. St. Louis, MO: Mosby;1991. 3. Gottrup F. A specialized wound-healing center concepts: importance of a multidisciplinary department structure and surgical treatment facilities in the treatment of chronic wounds. Am J Surg. 2004;187(Suppl):38S–43S. 4. Gottrup F. The EWMA patient outcome group. J Wound Care. 2009;18(11):460. 5. Gottrup F, Apelqvist J, Price P; European Wound Management Association Patient Outcome Group. Outcomes in controlled and comparative studies on non-healing wounds: recommendations to improve the quality of evidence in wound management. J Wound Care. 2010;19(6):237–268. 6. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clinical Pharmacol Ther. 2001;69(3):89–95. 7. Matousek S, Deva AK, Mani R. Outcome measurements in wound healing are not inclusive: a way forward. Int J Lower Extrem Wounds. 2007;6(4):284–290. 8. US Department of Health and Human Services, Food and Drug Administration. Guidance for Industry: Chronic Cutaneous Ulcer and Burn Wounds: Developing Products for Treatment. June 2000. 9. Hinman CD, Maibach H. Effect of air exposure and occlusion on experimental human skin wounds. Nature. 1963;200:377–378. 10. Gelfand JM, Hoffstad O, Margolis DJ. Surrogate endpoints for the treatment of venous leg ulcers. J Invest Dermatol. 2002;119(6):1420–1425. 11. Arnold TE, Stanley JC, Fellows EP, et al. Prospective, multicenter study of managing lower extremity venous ulcers. Ann Vasc Surg. 1994;8(4):356–362. 12. Romanelli M. Objective measurement of venous ulcer debridement and granulation with a skin color reflectance analyzer. WOUNDS. 1997;9(4):122–126. 13. Gardner SE, Frantz RA, Doebbling BN. The validity of the clinical signs and symptoms used to identify localized chronic wound infection. Wound Repair Regen. 2001;9(3):178–186. 14. Bolton LL. Evidence-based report card: operational definition of moist wound healing. J WOCN. 2007;34(1):23–29. 15. Cohn JN. Markers of malign across the cardiovascular continuum: interpretation and application. Introduction to surrogate markers. Circulation. 2004;109(25 Suppl 1):IV20–IV21. 16. ECRI Institute. Appendix B, Quality of Literature. ECRI Institute Study Quality Assessment Instrument, Negative Pressure Wound Therapy Devices. Available at: www.ecri.org. Other recommended literature can be found in the background article⁵ and at www.ewma.org/english/patient-outcome-group. Finn Gottrup is from the Bispebjerg University Hospital, Copenhagen, Denmark; Jan Apelqvist is from the Malmö University Hospital, Sweden; and Patricia Price is from the Cardiff University, United Kingdom. Address correspondence to: Finn Gottrup, MD, DMSci Professor of Surgery Copenhagen Wound Healing Center Department of Dermatology, D42 Bispebjerg University Hospital DK-2400 Copenhagen NV, Denmark fgottrup@post4.tele.dk