But it can be difficult to determine what qualifies as “strong” evidence when evaluating research. A recent article in Nursing 20211, the first of a three-part series, helps nurse leaders assess the quality, quantity, and consistency of research evidence to help determine its strength.
Three elements of strong research
According to the Agency for Healthcare Research and Quality, the strength of any research evidence must be determined using the following three elements:
- Quality – Perhaps the most challenging element nurses must evaluate, quality refers to the research methods used to ensure valid research results which are not influenced by bias or occurring by chance.
- Quantity – There must be a sufficient number of studies on the impact of a particular topic or treatment, and the studies must be of appropriate size.
- Consistency – Similar findings to the research study should be reported from other sources.
Evidence hierarchies: Understanding different approaches to research
Research evidence hierarchies can be thought of as pyramids made up of five separate levels. At the top of the pyramid, which can be thought of as Level 1, the evidence is strongest. The bottom of the pyramid, or Level 5, contains the weakest level of evidence for any given research topic.
Level 1: RCTs, systematic reviews, and meta-analyses
The strongest evidence for or against a new treatment or procedure is represented in Level 1, which evaluates the results from randomized controlled trials (RCTs), systematic reviews, and meta-analyses. In RCTs, studies must meet three criteria:
- Random assignment of study participants into two or more groups
- A treatment or procedure given to at least one of the groups
- A control group that doesn’t receive the new treatment or procedure
Systematic reviews use meticulous processes to identify, appraise, and synthesize evidence on a particular topic. Meta-analyses take this a step further by conducting statistical analyses of the synthesized data. This allows researchers to obtain statistics representing the effect of the treatment or procedure across multiple studies.
Level 2: Quasi-experimental research
The research results contained in Level 2 are termed “quasi-experimental” because, in general, the studies supplying the results lack one or two of the three criteria required for a true RCT design. Systematic reviews may be included in this level with or without meta-analysis of the data. There is still manipulation of study participants or the introduction of a new intervention, such as a new treatment.
Nurses should be more cautious when evaluating Level 2 experimental results, since reviews that include quasi-experimental research aren’t as strong as those including RCTs. But quasi-experimental research will still continue—in most cases, these types of studies are simpler to carry out.
Levels 3-5: Nonexperimental factors
The strength of research evidence continues to decline through levels three through five. Level 3 may be made up of nonexperimental study results which may or may not feature systematic reviews and meta-analysis. Level 4 is generally based on expert opinions which are, in turn, based on scientific evidence, practice guidelines, or consensus panels. Finally, Level 5 evidence usually comes from literature reviews or case reports, quality improvement programs or financial evaluations, or expert opinions based on experimental evidence.
The next article in this three-part series delves into the lower levels of the evidence hierarchy pyramid in greater detail. Still, nurses must rely on a sufficient number of sources to ultimately arrive at conclusions which influence nursing practice. It isn’t enough to find one piece of evidence that supports their suspicions—instead, critical thinking skills must be applied to determine the strength of evidence based on quality, quantity, and consistency of study results.
