Abstract: Poor medication adherence results in worse clinical outcomes and increased health care expenditures. Clinical pathways programs should address medication adherence to support the fundamental purpose of improving the quality of care. In order to develop initiatives to improve medication adherence, adherence must be measured using validated methods. A method’s validity depends on its accuracy in relation to a “gold standard,” the best measure available. The objective of this article is to review the strengths and limitations of commonly used methods for measuring adherence, describe the significance of a gold standard, and explore several candidates for this designation.
A primary goal of using clinical pathways is to improve the quality of care by promoting evidence-based treatment. Following evidence-based treatment plans requires the active participation of not only the provider but also the patient. Indeed, recognition of the importance of the patient’s role is one of the reasons why “medication adherence,” which suggests a collaborative relationship between the patient and the provider, has become the preferred term over “medication compliance,” which connotes a paternalistic sentiment.1
Research has shown that poor medication adherence results in worse clinical outcomes—including increased mortality and hospitalizations—leading to greater health care expenditures.2-14 A meta-analysis of 21 studies determined a strong association between appropriate adherence to treatment for a variety of diseases and lower mortality rates.11 This study also identified a strong “healthy adherer” effect—which accounts for the adoption of healthier lifestyles that often accompany adherent behaviors—on mortality. However, a separate study evaluating statin adherence post-myocardial infarction concluded that survival improves as a result of greater medication adherence independent of this healthy adherer effect.13 Hospitalization rates have also been found to be lower for patients with various chronic conditions who exhibit appropriate medication adherence.12,14 A review of 23 studies found that increased adherence to treatment for diabetes and cardiovascular diseases led to increased drug costs, but these increased costs were offset by reduced non-drug costs, leading to overall cost savings.7
Consistent with these effects, the 2003 World Health Organization (WHO) report regarding medication adherence declared that “increasing the effectiveness of adherence interventions may have a far greater impact on the health of the population than any improvement in specific medical treatment.”15 Therefore, the capacity for a clinical pathways program to achieve the goal of optimizing treatment becomes challenging if medication adherence is not addressed. Ideally, a clinical pathways program would incorporate a valid mechanism to identify patients with poor adherence and intervene in order to correct the root cause of the problem.
To develop and implement a successful intervention to improve medication adherence, it is imperative to measure adherence through validated and reliable methods. Each of the existing methods for measuring adherence exhibits certain strengths and limitations that should be considered prior to employing a given method and subsequently while interpreting results. The ultimate validation of an indirect method of determining medication adherence, and indeed a measurement of any clinical process, necessitates an identification of a “gold standard,” which can be concurrently compared to the measure of interest so as to ascertain its accuracy. A gold standard plays an imperative role in our ability to interpret results of measurements assessed in actual clinical practice and to subsequently translate and apply these results to clinical decision-making.16,17 Therefore, a choice of a highly flawed gold standard can produce many negative consequences by introducing bias to the validation process, including that for adherence measures. Depending on whether the gold standard overestimates or underestimates adherence and whether it has a tendency to miscalculate in the same direction as the measure of interest, the measure may be falsely validated.
The objective of this article is to describe the significance of a gold standard for measuring medication adherence and to explore several candidates for this designation, reviewing the strengths and limitations of commonly used methods for measuring adherence. Future applications of adherence measures in order to improve the outcomes of clinical pathways are also addressed.
MEASURES OF ADHERENCE
Methods for measuring medication adherence can be categorized into two basic types: direct measurement and indirect measurement. Direct measurement refers to the firsthand observation of drug administration or the detection of the drug or its metabolite in a biological fluid. Direct methods are considered to be more accurate than indirect methods; however, the complicated logistics of performing these measurements are an inherit disadvantage. In general, direct measurements are relatively costly and are more labor-intensive for the health care provider.18 Due to these disadvantages, it is unreasonable to use direct methods for measuring medication adherence in large patient populations. Indirect methods of measuring adherence are more commonly employed, due to their overall ease of use and less costly implementation.
Patient Self-Report
Self-reported measures, such as diaries and questionnaires, are fairly simple to administer. However, because some patients may be resistant to periodically answering a series of questions or entering their medication administration information in a diary on a regular basis, this may present an obstacle to the enrollment process. Additionally, because the results are derived from the patients’ self-reports, they are vulnerable to overestimation of adherence. Results may be easily manipulated by patients and are susceptible to recall bias, especially in the case of questionnaires. Although diaries are less influenced by recall bias, they have been used as an intervention to improve adherence and thus may reflect higher adherence rates than are typical for patients who do not track medication-taking using diaries.18-21
Pill Counting
Pill counting is one of the simplest methods to determine adherence among orally administered treatments. However, it requires that patients bring their entire medication supply to every visit with the health care provider and refrain from sabotaging the correct pill count by discarding unused medication instead of administering it as directed.19,20,22,23
Clinical Response
Similar to pill counts, the patients’ clinical response can be used as a surrogate marker of adherence. Measurement of the clinical response can be performed during regular visits to the health care provider and may already constitute the standard of care for monitoring of the disease being treated. While this method may allow the investigator to capture severe nonadherence, each patient’s individual clinical response to a medication is usually affected by a complex combination of variables, rendering a precise derivation of medication adherence very difficult.18
Refill Data
Unlike all of the previous methods discussed, the utilization of administrative refill data for medication adherence determination does not require patient participation and provides objective measurement of medication adherence in a naturalistic setting. This method also allows for an evaluation of a large number of patients over an extended period of time. However, the adherence value obtained from refill data does not produce any information on medication consumption; rather, it solely provides assessment of acquisition and possession of medication. It is assumed that patients administer the medication between the day of dispensation and the day of the refill. Since this method is based on prescription refills of prescriptions, it is better suited to study chronic rather than short-term treatment. Also, the use of refill data may be problematic for medications that require frequent dosage changes such as anticoagulants, anticonvulsants, and immunosuppressive medications. Finally, medications cannot be purchased outside of the closed system, where all prescription refills are documented in the same database.
Electronic Drug Monitoring
Electronic drug monitors, including the medication event monitoring system (MEMS), consist of specialized microchips incorporated into medication bottles that catalogue every opening of the bottle. MEMS offers a precise record of patients’ medication-taking behavior provided that each bottle-opening truly represents a single administration and that patients avoid transferring medications into other containers. These systems are also expensive and usually require regular downloading of information directly from the microchip to the compatible software program.19,23,24
Although it is generally acknowledged that a gold standard to measure medication adherence has not been identified by the virtue of a consensus from the scientific community involved in this area, MEMS has emerged as the de facto measure of choice to validate other adherence measures. There are several reasons for this recognition, one of which is simply the fact that MEMS consistently produces adherence rates below self-reported methods. Because self-reported methods are notorious for over-estimating adherence, it is assumed that MEMS is a superior estimate of true medication adherence behavior.25,26
In addition, medication adherence based on MEMS has been shown to closely correlate with clinical efficacy of different treatments in various therapeutic areas.27-29 For example, poor adherence to imatinib therapy, as measured by MEMS, for the treatment of chronic myeloid leukemia was observed to be significantly associated with inadequate molecular response.29 A study of human immunodeficiency virus (HIV) treatment concluded that MEMS was more sensitive in detecting nonadherence to antiretroviral therapy compared with self-reported methods. That study also observed a strong correlation of medication adherence based on MEMS with concurrent HIV viral load; most notably, the likelihood of actually achieving virologic suppression was more acutely associated with medication adherence based on MEMS versus self-reported methods.28
USING MULTIPLE ADHERENCE MEASURES
Although all of these methods are commonly used to measure adherence, due to their unique characteristics they often produce varying results. A comprehensive meta-analysis of studies comparing various measures of medication adherence found that less than half of self-reported measures, such as diaries and questionnaires, and non-self-reported measures, such as electronic monitors and refill data, were categorized as highly concordant.30 A cross-sectional study likewise found a weak concordance between measures using administrative refill records and patient-reported questionnaires.31 Consistently, studies have observed higher rates of medication adherence among self-reported measures, likely a product of overestimation.32-34
It has been suggested that utilizing more than one method of measuring adherence simultaneously increases the overall acuracy.35 This approach is based on the idea that the multiple methods for measuring adherence would complement each other by overcoming individual weakness of one measure with the strength of another. Akin to this concept is the composite reference standard (CRS), where the results of several imperfect tests are used to define a single reference standard. The premise relies on the assumption that the composite is more accurate than each of the tests making up CRS independently.36,37
CRS is actually a broad term encompassing all of the various approaches of combining multiple tests. Therefore, there is no set rule in terms of how the tests are incorporated. For example, in the case of tests with dichotomous results, an “or” rule can be used to indicate that any of the component tests must be positive to derive a positive result for the composite. Conversely, an “and” rule would stipulate that all of the component tests must be positive to conclude a positive result of the composite. For continuous measures, such as adherence rates, a composite score may be calculated.
However, a standardized method for calculating such a score has never been established. A study evaluating medication adherence of antiretroviral therapy in HIV patients compared the predictive strength of HIV viral load to adherence levels based on MEMS, pill count, interview questionnaire, and a composite adherence score calculated by with a combination of all three methods. The composite adherence score was calculated using a hierarchical algorithm. MEMS was the primary measure used to calculate the score; however, when MEMS values were missing or believed to be inaccurate, the score was calculated using pill counts. Subsequently, when pill counts were missing the score was calculated using calibrated values form the interview questionnaire. The study demonstrated that the composite score was the strongest predictor of undetectable viral load, followed by MEMS and pill counts alone.38 Another option when calculating a composite score is to average the standardized means of the component scores.
CRS is truly only useful if the various tests making up CRS itself correct the possible bias that each test exclusively manifests instead of amplifying the bias shared among the tests. This conditional dependence can be minimized by selecting component tests that evaluate different aspects of medication adherence—for example, drug levels and pill counts—or measures exhibiting different flaws—for example, questionnaires, which are susceptible to recall bias, in conjunction with refill data, which may overestimate adherence as a result of frequent dosing changes.38
Although CRS has not been widely used, it has the potential of emerging as the gold standard for measuring medication adherence by harnessing an already established idea of using multiple adherence measures to arrive to the most accurate results. However, for this to materialize, standardized CRS methods must be developed and successfully implemented in a variety of therapeutic areas.
STATISTICAL MODELING
In order to overcome the absence of a gold standard for certain diagnostic tests, special statistical methods have also been developed. Latent class models (LCMs) involve at least two diagnostic tests, both lacking established accuracies, applied to individuals from two or more populations. Through maximum likelihood procedures, sensitivities and specificities, as well as prevalence of the condition being measured in all populations, are estimated. LCMs may incorporate both classical frequentist generalized linear models or Bayesian statistics. The diagnostic tests may simply represent variables that are predictive of the outcome, also known as the latent variable.
A caveat of LCMs is that the latent variable, generally representing the disease status of the patient, must be categorical. In the context of medication adherence, the latent variable generally represents the absence or presence of nonadherence as determined by a pre-specified cutoff for the proportion of medications administered. To utilize LCM, several assumptions must also be met to achieve reliable results. First, the populations tested must exhibit different rates of the condition being measured; in other words, the rates of nonadherence have to be different between the groups. Another critical assumption is that results of the tests included in the model must be conditionally independent.39-41
This method is fairly complex and requires the use of specialized statistical modeling software, which may present a significant barrier to its use. As a result, LCM has been infrequently employed in the determination of medication adherence. However, its uncommon use in medication adherence research does not automatically invalidate its use. One retrospective study applied a LCM model to classify adherence to lipid-lowering, antihypertensive, and antidiabetic medications as “excellent, intermediate, and low” adherence. Patients achieving “excellent” adherence demonstrated significantly better serum profiles with fewer emergency department visits and hospitalizations.42 In the diagnostic test arena the use of LCMs has increased significantly, especially in the field of infectious diseases.39-41 Without more studies in the area of medication adherence, it would be premature to assume that LCMs could become a gold standard. Still, because the objective of using LCMs is to specifically address the lack of a gold standard, it corresponds well with one of the primary issues concerning the measurement of medication adherence.
CONCLUSION
Clinical pathways programs that involve self-administered medications should integrate medication adherence measurements as a part of their regularly analyzed outcomes. The choice of such a measure should be based on the feasibility of data collection in the context of the program itself and in consideration of the various strengths and weaknesses of the measure.
If accessible, administrative refill data would allow for an evaluation of medication adherence of all patients taking part in a pathway program. On the other hand, self-reported measures such as a questionnaire would promote patient engagement with not only the specific medication treatment itself but also with the pathways program overall. Above all, the selected measure must be validated relative to what can be regarded as a gold standard. While the accuracy of MEMS has been established based on a plethora of evidence, neither CRS nor LCM has been extensively studied in the area of medication adherence.
The label of a gold standard should not be misinterpreted as conferring an endorsement as the principal measure of adherence, which all studies evaluating this outcome should be expected to employ, or one that should be considered in clinical practice to identify nonadherent patients. Instead, it should simply be regarded as a useful tool to validate other more practical measures that can be effectively implemented in certain settings based on the particular advantages they may possess. The pursuit of further research in the utility of these measures to evaluate medication adherence is highly encouraged and may hold the key in identifying the ultimate gold standard.
References
1. American Medical Association. AMA Manual of Style: A Guide for Authors and Editors, 10th ed. New York: Oxford University Press, Inc; 2007.
2. Senst BL, Achusim LE, Genest RP, et al. Practical approach to determining costs and frequency of adverse drug events in a health care network. Am J Health Syst Pharm. 2001;58(12):1126-1132.
3. Schiff GD, Fung S, Speroff T, McNutt RA. Decompensated heart failure: Symptoms, patterns of onset, and contributing factors. Am J Med. 2003;114(8):625-630.
4. Rodgers PT, Ruffin DM. Medication nonadherence—part II: a pilot study in patients with congestive heart failure. Manag Care Interface. 1998;11(9):67-69, 75.
5. McDonnell PJ, Jacobs MR. Hospital admissions resulting from preventable adverse drug reactions. Ann Pharmacother. 2002;36(9):1331-1336.
6. Misdrahi D, Llorca PM, Lançon C, Bayle FJ. Compliance in schizophrenia: predictive factors, therapeutical considerations and research implications [article in French]. Encephale. 2002;28(3 Pt 1):266-272.
7. Muszbek N, Brixner D, Benedict A, Keskinaslan A, Khan ZM. The economic consequences of noncompliance in cardiovascular disease and related conditions: a literature review. Int J Clin Pract. 2008;62(2):338-351.
8. Rodgers PT, Ruffin DM. Medication nonadherence—part I: the health and humanistic consequences. Manag Care Interface. 1998;11(8):58-60.
9. Cramer JA, Benedict A, Muszbek N, Keskinaslan A, Khan ZM. The significance of compliance and persistence in the treatment of diabetes, hypertension and dyslipidaemia: a review. Int J Clin Pract. 2008;62(1):76-87.
10. Cleemput I, Kesteloot K, DeGeest S. A review of the literature on the economics of noncompliance. Room for methodological improvement. Health Policy. 2002;59(1):65-94.
11. Simpson SH, Eurich DT, Majumdar SR, et al. A meta-analysis of the association between adherence to drug therapy and mortality. BMJ. 2006;333(7557):15.
12. Sokol MC, McGuigan KA, Verbrugge RR, Epstein RS. Impact of medication adherence on hospitalization risk and healthcare cost. Med Care. 2005;43(6):521-530.
13. Rasmussen JN, Chong A, Alter DA. Relationship between adherence to evidence-based pharmacotherapy and long-term mortality after acute myocardial infarction. JAMA. 2007;297(2):177-186.
14. Ho PM, Magid DJ, Shetterly SM, et al. Medication nonadherence is associated with a broad range of adverse outcomes in patients with coronary artery disease. Am Heart J. 2008;155(4):772-779.
15. World Health Organization, Sabaté E. Adherence to long-term therapies: Evidence for action. Geneva: World Health Organization; 2003:198.
16. Johnson WO, Gastwirth JL, Pearson LM. Screening without a “gold standard”: The Hui-Walter paradigm revisited. Am J Epidemiol. 2001;153(9):921-924.
17. Waikar SS, Betensky RA, Emerson SC, Bonventre JV. Imperfect gold standards for kidney injury biomarker evaluation. J Am Soc Nephrol. 2012;23(1):13-21.
18. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487-497.
19. Partridge AH, Avorn J, Wang PS, Winer EP. Adherence to therapy with oral antineoplastic agents. J Natl Cancer Inst. 2002;94(9):652-661.
20. Ruddy K, Mayer E, Partridge A. Patient adherence and persistence with oral anticancer treatment. CA Cancer J Clin. 2009;59(1):56-66.
21. Wang PS, Benner JS, Glynn RJ, Winkelmayer WC, Mogun H, Avorn J. How well do patients report noncompliance with antihypertensive medications?: a comparison of self-report versus filled prescriptions. Pharmacoepidemiol Drug Saf. 2004;13(1):11-19.
22. Pullar T, Kumar S, Tindall H, Feely M. Time to stop counting the tablets? Clin Pharmacol Ther. 1989;46(2):163-168.
23. Rudd P, Byyny RL, Zachary V, et al. The natural history of medication compliance in a drug trial: limitations of pill counts. Clin Pharmacol Ther. 1989;46(2):169-176.
24. Hansen RA, Kim MM, Song L, Tu W, Wu J, Murray MD. Comparison of methods to assess medication adherence and classify nonadherence. Ann Pharmacother. 2009;43(3):413-422.
25. Wilson IB, Carter AE, Berg KM. Improving the self-report of HIV antiretroviral medication adherence: Is the glass half full or half empty? Curr HIV/AIDS Rep. 2009;6(4):177-186.
26. Vriesendorp R, Cohen A, Kristanto P, et al. Adherence to HAART therapy measured by electronic monitoring in newly diagnosed HIV patients in Botswana. Eur J Clin Pharmacol. 2007;63(12):1115-1121.
27. Acosta FJ, Bosch E, Sarmiento G, Juanes N, Caballero-Hidalgo A, Mayans T. Evaluation of noncompliance in schizophrenia patients using electronic monitoring (MEMS) and its relationship to sociodemographic, clinical and psychopathological variables. Schizophr Res. 2009;107(2-3):213-217.
28. Arnsten JH, Demas PA, Farzadegan H, et al. Antiretroviral therapy adherence and viral suppression in HIV-infected drug users: comparison of self-report and electronic monitoring. Clin Infect Dis. 2001;33(8):1417-1423.
29. Marin D, Bazeos A, Mahon FX, et al. Adherence is the critical factor for achieving molecular responses in patients with chronic myeloid leukemia who achieve complete cytogenetic responses on imatinib. J Clin Oncol. 2010;28(14):2381-2388.
30. Garber MC, Nau DP, Erickson SR, Aikens JE, Lawrence JB. The concordance of self-report with other measures of medication adherence: a summary of the literature. Med Care. 2004;42(7):649-652.
31. Cook CL, Wade WE, Martin BC, Perri M III. Concordance among three self-reported measures of medication adherence and pharmacy refill records. J Am Pharm Assoc (2003). 2005;45(2):151-159.
32. Farmer KC. Methods for measuring and monitoring medication regimen adherence in clinical trials and clinical practice. Clin Ther. 1999;21(6):1074-90; discussion 1073.
33. George J, Kong DC, Stewart K. Adherence to disease management programs in patients with COPD. Int J Chron Obstruct Pulmon Dis. 2007;2(3):253-262.
34. Turner BJ. Adherence to antiretroviral therapy by human immunodeficiency virus-infected patients. J Infect Dis. 2002;185 (suppl 2):S143-S151.
35. Lam WY, Fresco P. Medication adherence measures: an overview. Biomed Res Int. 2015;2015:217047.
36. Alonzo TA, Pepe MS. Using a combination of reference tests to assess the accuracy of a new diagnostic test. Stat Med. 1999;18(22):2987-3003.
37. Naaktgeboren CA, Bertens LC, van Smeden M, de Groot JA, Moons KG, Reitsma JB. Value of composite reference standards in diagnostic research. BMJ. 2013;347:f5605.
38. Liu H, Golin CE, Miller LG, et al. A comparison study of multiple measures of adherence to HIV protease inhibitors. Ann Intern Med. 2001;134(10):968-977.
39. Hartnack S, Budke CM, Craig PS, et al. Latent-class methods to evaluate diagnostics tests for Echinococcus infections in dogs. PLoS Negl Trop Dis. 2013;7(2):e2068.
40. Rindskopf D, Rindskopf W. The value of latent class analysis in medical diagnosis. Stat Med. 1986;5(1):21-27.
41. van Smeden M, Naaktgeboren CA, Reitsma JB, Moons KG, de Groot JA. Latent class models in diagnostic studies when there is no reference standard—a systematic review. Am J Epidemiol. 2014;179(4):423-431.
42. Davis J, Starr R, Juarez DT. Evidence-based adherence classes for combination antihypertensive, lipid-lowering, and antidiabetic therapy. Am J Pharm Benefits. 2015;7(4):e108-e115.
