When it comes to evaluating the economic impact of projects and interventions, there are five critical success factors to take into account.
It’s advisable to adopt a systematic approach that carefully identifies those inputs that directly influence measurable outputs and the economic outcomes they drive. Understanding these inputs and the outputs then achieved helps to accurately measure the economic effects of initiatives and allows organisations to make informed future decisions based on reliable data.
In this blog post, we will explore five key elements that play a pivotal role in assessing economic impact and ensuring the validity of impact reports.
1. Relevant and Reasonable Outputs
For successful assessment of economic impact, it's crucial to ensure that the outputs being measured are both relevant and reasonable. The impacts should be clear, direct, and causally linked to the project in question. Often direct impacts are then muddled with additional indirect outputs being brought into the calculation, but caution is needed here. Capturing indirect outputs can be achieved through the use of multipliers, which help account for the ripple effects of economic activities. Multipliers offer a way to estimate how changes in one sector can lead to changes in other, related sectors, providing a more comprehensive view of the overall economic impact. Capturing indirect impact and also applying a multiplier leads to double counting and as such undermines the whole exercise.
2. Benefits Attribution
Attributing benefits accurately requires careful consideration of various factors, including the organisations involved, the geographical scope, and the duration of the impact. Identifying those directly benefiting from the project is vital, as it contributes to a more precise measurement of economic impact. Additionally, considering the persistence effect—the duration over which the benefits will continue—is essential for understanding the long-term implications of the intervention. In many cases, the market may address issues over the medium term / or an alternative approach is found in the market – and as such the persistent effect should always be considered over a finite period. The geographical area considered tends to be a political consideration – Local Authorities will tend to want to understand the impact on their geographic boundary for example.
3. Additionality
Another critical aspect of measuring economic impact is the concept of additionality. Projects and interventions should generate economic impacts that go beyond what would have occurred naturally or without the intervention. In other words, the impact assessment should focus on measuring the net change brought about by the project. This approach helps to avoid overestimating the effects and provides a more accurate representation of the project's actual contribution to the economy.
The default position of many evaluations is that nothing would take place without the project progressing and this is rarely the case in the real world. Understanding what might have happened, however, requires some careful use of control groups or undertaking trend analysis if it is to be completed properly.
4. Gross to Net Analysis
To create a robust impact report, it's essential to perform a thorough analysis that validates assumptions around leakage, displacement, and economic multipliers. Leakage occurs when the economic benefits of a project "leak" out of the targeted area or community. Displacement happens when the project merely shifts economic activity from one place to another without generating new value. A comprehensive assessment considers these factors and adjusts the gross impact to arrive at a more accurate net impact figure. Displacement may not simply relate to the type of intervention considered – in an economy at virtually full employment there is likely to be significant displacement from different areas of the economy when an intervention takes place.
5. Managing Optimism Bias
Optimism bias is a common challenge in economic impact assessments. It involves the tendency to assume that cost estimates will be accurate, projects will be completed on time, and desired outputs will be achieved without setbacks. To counteract this bias, it's crucial to temper the results with a healthy dose of realism. Using sensitivity analysis—testing the impact of different scenarios and assumptions—can provide a more balanced view of potential outcomes and mitigate the effects of overly optimistic projections. Supplementary Green Book documents also provide survey information on what has happened to real projects over time.
Accurately assessing economic impact calls for a thoughtful and systematic approach that considers the various factors that can influence outcomes.
By focusing on relevant outputs, careful benefit attribution, additionality, gross to net analysis, and managing optimism bias, organisations can ensure that impact reports provide a reliable basis for decision-making.
Time taken in examining causal chains of impact helps in not only measuring economic impact more accurately, but also in demonstrating a commitment to transparency and informed decision-making. The same methodology is also essential in building the economic case for Treasury Green Book business cases.
If you would like to find out more about best practice in developing or finessing an economic strategy, please contact us.
Written by Lois McLuckie, Marketing Associate