User:Jslacy/sandbox

The National Registry of Evidence-based Programs and Practices (NREPP) is an online, searchable database of interventions designed to promote mental health or prevent or treat substance abuse and mental disorders. NREPP is a public service funded and administered by the Substance Abuse and Mental Health Services Administration (SAMHSA), part of the U.S. Department of Health and Human Services. The goal of the Registry is to encourage wider adoption of evidence-based interventions and to help those who are interested in implementing an evidence-based intervention to select one that best meets their needs.

Overview

Logo: SAMHSA's National Registry of Evidence-based Programs and Practices (NREPP), www.nrepp.samhsa.gov

In the behavioral health field, there is an ongoing need for researchers, developers, evaluators, and practitioners to share information about what works to improve outcomes among individuals coping with, or at risk for, mental disorders and substance abuse. Discussing how this need led to the development of NREPP, Brounstein, Gardner, and Backer (2006) write:

[I]t is important to note that not all prevention programs work. Still other programs have no empirically based support regarding their effectiveness. … Many others have empirical support, but the methods used to generate that support are suspect. This is another reason to highlight the need for and use of scientifically defensible, effective prevention programs. These are programs that clearly demonstrate that the program was well implemented, well evaluated, and produced a consistent pattern of positive results.

The focus of NREPP today is on delivering an array of standardized, comparable information on interventions that are evidence based, as opposed to identifying programs that are “effective” or ranking them in effectiveness. NREPP’s peer reviewers use specific criteria to rate the quality of an intervention’s evidence base (so that developers’ claims of effectiveness can more accurately be judged) as well as the intervention’s suitability for broad adoption. In addition, NREPP provides contextual information about the intervention, such as the population served, implementation history, and cost data to encourage a realistic and holistic approach to selecting prevention interventions.

As of 2010, the interventions reviewed by NREPP have been implemented successfully in more than 229,000 sites, in all 50 States and more than 70 countries, and with more than 107 million clients. Versions of the NREPP review process and rating criteria have been adopted by two other agencies in the U.S. Department of Health and Human Services, the National Cancer Institute and the Administration on Aging.

The information NREPP provides is subject to certain limitations that are explained on its Web site. One limitation is that NREPP is not an exhaustive repository of all tested mental health interventions; submission is a voluntary process, and limited resources may preclude the review of some interventions even though they meet minimum requirements for acceptance. Readers are directed to contact intervention developers to get more information before deciding whether to purchase or implement a particular intervention. The NREPP home page prominently states that “inclusion in the registry does not constitute an endorsement.”

Submission Process

NREPP holds an open submission period that runs November 1 through February 1. Open submission periods have been announced on the NREPP Web site and in the Federal Register. For an intervention to be eligible for a review, it must meet four minimum criteria:

1. The intervention has produced one or more positive behavioral outcomes (p ≤ .05) in mental health, mental disorders, substance abuse, or substance use disorders use among individuals, communities, or populations.
2. Evidence of these outcomes has been demonstrated in at least one study using an experimental or quasi-experimental design.
3. The results of these studies have been published in a peer-reviewed journal or other professional publication, or documented in a comprehensive evaluation report.
4. Implementation materials, training and support resources, and quality assurance procedures have been developed and are ready for use by the public.

SAMHSA selects the interventions to be reviewed from the submissions received that meet these minimum criteria. In some years, additional criteria relating to the prioritization and selection of eligible interventions has been provided in the Federal Register announcements. Once reviewed and added to the Registry, interventions are invited to undergo a new review 4 or 5 years after their initial review.

Review Process

The NREPP review process consists of two parallel and simultaneous review tracks, one that looks at the intervention’s Quality of Research and another that looks at the intervention’s Readiness for Dissemination. The materials used in a QOR review are generally published research articles, although unpublished final evaluation reports can also be included. The materials used in an RFD review include implementation materials and process documentation, such as manuals, curricula, training materials, and written quality assurance procedures.

The reviews are conducted by expert consultants who have received training on NREPP's review process and rating criteria. Two QOR and two RFD reviewers are assigned to each review. Reviewers work independently, rating the same materials. Their ratings are averaged to generate final scores.

While the review process is ongoing, NREPP staff work with the intervention’s representatives to collect descriptive information about the intervention, such as the program goals, types of populations served, and implementation history.

Rating Criteria

The rating criteria used in NREPP reviews are explained on the NREPP Web site. Several articles and other sources have described the supporting rationale for these criteria and how they were developed. These criteria and the accompanying rating anchors are unique to NREPP but share common elements with the types of standards used by other Federal agencies to assess evidence-based programs.

The QOR ratings, given on a scale of 0.0 to 4.0, indicate the strength of the evidence supporting the outcomes of the intervention. Higher scores indicate stronger, more compelling evidence. Each outcome is rated separately because interventions may target multiple outcomes (e.g., alcohol use, marijuana use, behavior problems in school), and the evidence supporting the different outcomes may vary. The QOR rating criteria are:

1. Reliability of measures
2. Validity of measures
3. Intervention fidelity
4. Missing data and attrition
5. Potential confounding variables
6. Appropriateness of analysis

The RFD ratings, also given on a scale of 0.0 to 4.0, indicate the amount and quality of the resources available to support the use of the intervention. Higher scores indicate that resources are readily available and of high quality. These ratings apply to the intervention as a whole. The RFD criteria are:

1. Availability of implementation materials
2. Availability of training and support resources
3. Availability of quality assurance procedures

Reviewer Recruitment and Qualifications

The NREPP Web site provides information on how reviewers are recruited and trained. QOR reviewers are required to have a doctoral-level degree and a strong background and understanding of current methods of evaluating prevention and treatment interventions. Individuals with significant knowledge of prevention and/or treatment services in substance abuse and mental health are preferred. RFD reviewers are selected from two categories: direct services experts (including both providers and consumers of services), or experts in the field of implementation. Direct service experts must have previous experience evaluating prevention or treatment interventions and knowledge of mental health or substance abuse prevention or treatment content areas. Implementation experts are required to have previous experience implementing interventions, doing evaluation work in service settings, and/or conducting research across interventions, as well as knowledge of mental health or substance abuse prevention or treatment content areas.

Reviewers are trained on the procedures and criteria they will use to rate interventions and are required to sign a Conflict of Interest form for each intervention they review to ensure they have no professional ties or financial or other interests in the intervention that could prevent an objective review. The identity of all NREPP reviewers assigned to specific reviews is kept confidential.

Products and Publications

NREPP publishes an intervention summary for each intervention it adds to the Registry. The summaries, which are accessed through the Registry’s search engine, contain the following standardized information:

• A brief description of the reviewed intervention, including targeted goals and theoretical basis
• Study populations (age, gender, race/ethnicity)
• Study settings and geographical locations
• Implementation history
• Funding information
• Comparative evaluation research conducted with the intervention
• Adaptations
• Adverse effects
• List of studies and materials reviewed
• List of outcomes
• Description of measures and key findings for each outcome
• Research design of the studies reviewed
• Quality of Research and Readiness for Dissemination ratings
• Reviewer comments (Strengths and Weaknesses)
• Costs
• Replication studies
• Contact information

NREPP also maintains an online Learning Center. Offerings include learning modules on implementation and preparing for NREPP submission; a research paper on evidence-based therapy relationships; and links to screening and assessment tools for mental health and substance use.

Predecessor System

The registry originated in 1997 and has gone through several changes since then. The predecessor to today's NREPP was the National Registry of Effective Prevention Programs (later renamed the National Registry of Effective Programs and Practices), which was developed by SAMHSA's Center for Substance Abuse Prevention as part of the Model Programs initiative. Procedures under this earlier registry were developed to review, rate, and designate programs as Model, Effective, or Promising. Based on extensive input from scientific communities, service providers, expert panels, and the public, the procedures were revised. Reviews using the new NREPP system began in 2006, and the redesigned Web site debuted in March 2007.

External links

• Official Web site
• Substance Abuse and Mental Health Services (SAMHSA)

References

[Category:Substance Abuse and Mental Health Services Administration]] [Category:Evidence-based practices]]