5. Evaluation design

Despite evidence (from the process evaluation) of the planned activity taking place at local level, we were not able to show that CICA was effective, nor was it cost-effective. There are likely to be two main reasons for this. 

Firstly, the number of champions trained was too small to have a sufficient number of brief advice conversations to generate any measurable effect at the area level of analysis. Moreover, brief interventions may not directly impact on primary health and crime outcomes at a population level. Alternative study designs could be sought.

Secondly, AHCs were less willing (and had less opportunity) to get involved with alcohol licensing decisions. Since licensing policies theoretically have an impact at an area level, this theoretically would have been more likely to generate a significant area-level effect. The evidence from this study is that communities continue to struggle to influence statutory processes that affect alcohol availability where they live, and further consideration of how to enable increased community engagement is necessary.

Recommendations for future research (numbered in priority order)

1. Natural experiment evaluations should include methodological triangulation to guard against overinterpretation of spurious results.
2. Investigate coproducing a community outcomes framework to measure reductions in alcohol harm.
3. Evaluate whether rebalancing local licensing policy to be community-centred might increase community engagement in the local licensing decision-making process.

4. Investigate the effectiveness of CICA in areas where indicators of alcohol harm are increasing but are not at the highest level of harm within the local authority.

5. A wider range of external contextual factors could be tested in the pre-implementation phase of a complex intervention, incorporating methods to combine factors into mean scores, to test relationships between process indicators and outcomes.

Alcohol health champions, CICA coordinators, RSPH trainers and commissioners share their experiences in this 16 minute documentary.

Where to find published research findings

Scott LJ, Hidajat M, Burns EJ, Ure C, Hargreaves SC, Audrey S, et al. Does a local Alcohol Health Champion programme have a measurable impact on health and crime outcomes? A natural experiment evaluation of Communities in Charge of Alcohol (CICA) based on triangulation of methods. Addiction 2023; https://doi.org/10.1111/add.16363

Burns EJ, de Vocht F, Teixeira Siqueira N, Ure C, Audrey S, Coffey M, et al. An ‘alcohol health champions’ intervention to reduce alcohol harm in local communities: a mixed-methods evaluation of a natural experiment. Public Health Research (in press)