Methodology

For each intervention listed on Reducing Reoffending, one of two methods has been used to systematically identify, evaluate and summarise the published evidence which describes the intervention’s effect on reoffending.

Where there is a relevant systematic review of an intervention, a summary of the existing systematic review has been produced. Relevant systematic reviews are those which include evidence from UK and European studies, have been undertaken recently (ideally within the last 5 years) and where the included studies are similar to those used in the UK criminal justice system.

For interventions for which no relevant systematic review has been identified, a Rapid Evidence Assessment (REA) has been undertaken. Rapid Evidence Assessments are a form of systematic review, and are undertaken over a shorter period (approximately 3 months, rather than 12 months) than a traditional systematic review. REAs and systematic reviews systematically search for, evaluate and synthesise evidence about a specific intervention, and they are used to help policymakers to understand the impact of an intervention. Where possible an REA or systematic review will include a statistical meta-analysis of individual studies, in order to provide a clear indication of the likely impact (effect size) of the intervention.

Reducing Reoffending Interventions

More Information

Reducing Reoffending Methodology

More Information

A Guide to the Rating Scales

The rating scales used on Reducing Reoffending allow users to filter interventions by different criteria: the category (type of intervention); the estimated impact of the intervention; the certainty of the estimated impact; the cost and ease of implementation of the intervention. Some of the rating scales have been adapted from existing well-validated scales (i.e. rating scales which have been found to be accurate and effective measures).

Average Impact

This scale indicates the impact of the intervention on reducing reoffending (the effect size). More detailed information about the intervention’s effect size can be found in the corresponding intervention summary (in the section ‘Does it work?’).

Positive effect Intervention reduces reoffending
No effect Intervention has no effect on reoffending
Negative effect Intervention increase reoffending
Effectiveness not measured The effect of an intervention was not measured

Average Impact

This scale tells us how confident we can be that the average impact (effect) identified by the reviewed studies estimates the ‘true’ effect. For summaries of systematic reviews it is derived from the authors’ quality ratings. For REAs it is derived from the transparent and systematic framework provided by the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (Balshem et al. 2011). The true effect is the actual impact of the intervention on reducing reoffending. Research attempts to measure the true effect of the intervention, but practical considerations mean that may not be possible for researchers to use research designs, data collection and analysis methods which most accurately measure the true effect. For example, a Randomised Controlled Trial (RCT) is a research design which can clearly help researchers to understand what is causing reductions (or increases) in reoffending. However it is often difficult (and ethically questionable) to randomly allocate individuals to receive an intervention (or not to). This means what researchers may need to use a non-randomised research design instead.

If the summarised evidence contains studies which are rated below ‘High’, we cannot be confident that the estimate of the effect (the average impact) is a good estimate of the true effect. The ratings for the summarised evidence are:

High We are very confident that the true effect lies close to that of the estimate of the effect
Moderate We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different
Low Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect
Very low We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect

Cost

This scale is intended to provide an indication of the financial cost associated with an intervention. It is anticipated that a number of studies will be based in the United States and/or will have been implemented several years ago. For this reason the costs presented here are indicative only.

£££££ High cost
££££ High–moderate cost
£££ Moderate cost
££ Low–moderate cost
£ Low cost
Not evaluated The evidence reviewed did not include an economic evaluation

System readiness

This scale is an indicative estimated measure of the ease of implementation for particular interventions, and is loosely based on Dartington Social Research Unit’s own measure. The scale indicates whether the intervention is accompanied by the necessary support and information to enable its implementation in service systems. For example, interventions with a high degree of system readiness meet criteria such as the following (Dartington Social Research Unit, 2011):

  • The intervention has been tested in ‘real world’ conditions.
  • Technical support is available to help implement the intervention in new settings.
  • There is a fidelity protocol or assessment checklist to accompany the intervention.
  • There are explicit processes for ensuring that the intervention gets to the right people.
  • There are training materials and implementation procedures.
  • There is reported information on the financial and human resources required to deliver the intervention.
5 High system readiness
4 High-moderate system readiness
3 Moderate system readiness
2 Low-moderate system readiness
1 Low system readiness

References

  1. Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Guyatt, G. H. (2011). GRADE guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology, 64(4), 401–406. https://doi.org/10.1016/J.JCLINEPI.2010.07.015
  2. Dartington Social Research Unit (2011). Programmes to promote the social and emotional well-being of vulnerable children aged 0-5 years: Summary of information from application of the Evidence2Success standards of evidence. The Social Research Unit, Dartington, United Kingdom.
  3. Mackenzie, D. L., Criminol, J. E., & Farrington, D. P. (n.d.). Preventing future offending of delinquents and offenders: what have we learned from experiments and meta-analyses? https://doi.org/10.1007/s11292-015-9244-9