The first day I wheeled through the doors of my public health fellowship at Kaiser Permanente Colorado Institute for Health Research, I was immersed in an environment that used evidence-based research to guide its clinical practice, which is a process known as translational research. In fact, my fellowship site mentor, Paul Estabrooks, greeted me with posters, surveys, and a research protocol. He said, “Go embed yourself at the clinic, engage the staff, and get the research project up and running….by noon.” Well, it didn’t all happen by noon, but I was on the road and heading toward the clinic to meet with staff and “embed” myself into the clinical processes.
RE-AIM Framework: Tool for Research and Evaluation
Our methodological approach applies research results to complex “real world” settings. What really helped me conceptualize this research was a framework developed by another mentor, Russ Glasgow, called the RE-AIM framework. RE-AIMis an acronym that stands for Reach, Effectiveness, Adoption, Implementation, and Maintenance. Since its development 18 years ago, the RE-AIM framework has grown and is now used by a diverse group of researchers and practitioners. The RE-AIM framework is meant to be used at all stages of research — from the beginning stages of planning through evaluation and reporting.
You might wonder why planning for implementation at the beginning of your research would be important to the research you conduct. Surprisingly, only 14 percent of evidence-based research transfers into population based-practice and this translation process takes, on average, 17 years to complete. But the RE-AIM framework mediates this lag by providing a sequenced approach to measure and evaluate outcomes along the research continuum, allowing for practical application of research in real world settings.
Use of RE-AIM Framework
At Kaiser Permanente, we have used the RE-AIM framework to guide research planning, implementation, and dissemination of asthma reminders, diabetes interventions, child obesity, smoking cessation, and online health interventions. More recently, we have used RE-AIM in grant management, community-based multilevel interventions, reduction of health disparities, and policy. RE-AIM also helps us understand the resources (such as staff) and costs that it takes to complete the research project.
The RE-AIM framework enables researchers to emphasize five dimensions that are often overlooked, but together determine public health impact; place equal emphasis on external and internal validity; and evaluate who and which settings participate in the program/research and who benefits to address health equity. RE-AIM is an evolving framework that adapts to current needs, while maintaining essential elements of structure over time. The framework also has pragmatic applications, and has become more accessible to non-researchers, such as community and clinical leaders.
The Five Dimensions of RE-AIM and their Definitions
|RE-AIM Dimension||Key questions to consider and answer|
|REACH your intended target population||WHO is/was intended to benefit, and who actually participates or is exposed to the intervention?|
|EFFICACY or EFFECTIVENESS||WHAT are/were the most important benefits you are/were trying to achieve and what is/was the likelihood of negative outcomes?|
|ADOPTION by target staff, settings, or institutions||WHERE is/was the program or policy applied, and WHO applied it?|
|IMPLEMENTATION consistency, costs, and adaptations made during delivery||HOW consistently is/was the program or policy delivered, HOW will it be/was it adapted, HOW much will/did it cost, and HOW/WHY will/did the results come about?|
|MAINTENANCE of intervention effects in individuals and settings over time||WHEN will (was) the initiative become operational; how long will (was) it sustained (Setting level); and how long are the results sustained (Individual level)?|
Translating Research into Practice through RE-AIM
|RE-AIM Dimension||Application to Influenza Vaccine Reminder Research|
|REACH your intended target population||All adults aged 19 to 64 years at KPCO with a diagnosis of asthma or COPD were identified (12,428). 143 were excluded due to lack of contact information. Demographic and clinical characteristics were reported between the three study arms.|
|EFFICACY or EFFECTIVENESS||Influenza vaccination rates were 29.5%, 31.1%, and 30.6% in the postcard-only, Interactive Voice Response (IVR)-only, and postcard plus- IVR study arms, respectively. After controlling for relevant covariates, IVR reminders were not significantly more or less effective than postcard reminders. Nationally, during the 2012-2013 influenza season, an estimated 47% of adults aged 19 to 64 years with chronic medical conditions were immunized against influenza, and 34.6% of adults with asthma aged 18 to 49 years were immunized.|
|ADOPTION by target staff, settings, or institutions||There was only one study site, so adoption was 100%.|
|IMPLEMENTATION consistency, costs, and adaptations made during delivery||For subjects receiving an IVR call, 57% received a message on their answering machine; 27% answered the call; and 16% were not reached. Program costs were $0.78, $1.23, and $1.93 per subject for postcard-only, IVR-only, and postcard-plus-IVR reminders, respectively. Extrapolating costs to the entire population at the study site that typically receives influenza vaccination reminders (approximately 100,000 individuals), reminder costs would have been $0.55, $0.05, and $0.60 per subject for postcard-only, IVR-only, and postcard-plus-IVR reminders, respectively. Over the years, the reminders were adapted to use text messaging and to follow up after a month if there was no receipt of influenza vaccine.|
|MAINTENANCE of intervention effects in individuals and settings over time||Based on study findings, IVR was adopted as the primary strategy for annual influenza vaccination reminders at the study site and continues with adaptations to meet changing technological advances, such as text messaging.|
In addition, I have used the RE-AIM framework for research planning. For example, I’ve used it in planning to implement a randomized control trial for asthma inhaler overuse (e.g., determining who would receive the intervention). I’ve also used the RE-AIM framework to evaluate research activities during the process (e.g., measuring the consistency of the intervention components across time), and then to assess whether the research was put into practice or policy (e.g., if effective, which venues continued the intervention).
As the health environment continues to change, questions such as “which complex intervention for what type of patient, delivered by what type of staff, in which settings, will be most cost effective, under which conditions and for what outcomes” will need to be answered. RE-AIM provides a structured method to help researchers answer these questions.