Step 7: Meta-analysis

Narrative summary techniques can be used to synthesise information regarding study type, animal population characteristics, study quality, interventions, and the outcomes measured. Narrative analysis adopts a textual approach, describing the relationships within and between studies and an overall assessment of the robustness of the evidence.

Meta-analysis, a quantitative statistical technique to combine the results from multiple studies, can be performed if your data are suitable. Results from diverse types of studies are not always suitable for pooling together and if limited data are available meta-analysis is not always appropriate.

Narrative and quantitative analyses are not mutually exclusive and aspects from narrative and meta-analyses can successfully be integrated into a systematic review.

 

Meta-analysis: Meta-analysis combines the results from individual studies to increase power and precision in estimating the overall intervention effects. Meta-analysis is performed in two-steps: Firstly, summary statistics for each individual study are calculated. Secondly, the individual study statistics are combined in a weighted average to give an overall estimate of intervention effect.

There are two models generally used to combine individual study statistics: fixed-effect and random-effects. These models represent fundamentally different assumptions regarding the underlying data. A fixed-effect model assumes one true effect size and so takes into account only within-study variation. A random-effects model allows for the true effect size to differ between studies, and takes into account both within- and between-study variations.

 

Investigating heterogeneity: Between-study variation is known as heterogeneity. Quantifying and exploring sources of heterogeneity are particularly important aspects of preclinical meta-analysis. A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. Common test statistics to quantify heterogeneity include Cochran’s Q and I2.

The sources of any heterogeneity can be investigated using stratified meta-analysis or meta-regression. Study design and quality characteristics that influence outcome, or aspects of treatment delivery that provide maximum efficacy can be identified.

 

Publication Bias: Publication bias occurs when the results of the published literature differ from the results of the entire pool of research conducted in that field. This can happen if research that is statistically significant is published while neutral results remain unpublished. More broadly, publication bias can encompass other dissemination biases for example selectively reporting only significant outcomes.

These biases mean that consumers of published science are at risk of drawing incorrect conclusions. We can assess and correct for publication bias using statistical methods including funnel plots, Egger regression and trim and fill. These tests should be used and interpreted with caution as their power is often low and factors other than publication bias can influence the results.

 

Further Information:

Vesterinen et al (2014) Meta-analysis of data from animal studies: A practical guide

Borenstein et al (2009) Introduction to Meta-Analysis

Cooper et al (2009) The Handbook of Research Synthesis and Meta-Analysis

Borenstein et al (2007) Meta-Analysis: Fixed effect vs. random effects

Frameworks for narrative synthesis can be found here

 

  < <  Move back to Step 6   Move on to Step 8  > >