STEP SEVEN: LEARNING FROM CHANGE

Monitoring and Evaluation

​

During the implementation of a project, it is important to monitor how well the process of change is being managed. If concerns are identified, changes can be made to improve the design and effectiveness of a project. Furthermore, if at the end of a project it is determined that the change process has not been as successful as planned, on-going monitoring and review undertaken during the implementation of the project can help an evaluation of why the change process may have failed to deliver fully on its original purpose.

​

Following the implementation of a project, it is important to evaluate whether the outcomes delivered have succeeded in bringing about the desired changes and benefits defined in the original project purpose and the goals and objectives set out in any underlying strategy.

​

An equally important part of the monitoring and evaluation process is to help consolidate what has been learnt from any change to help keep the process of change moving forward and deliver improvements in the change process in the future. For example, if as the result of any evaluation it is concluded that the original strategy needs to be revised, this will enable project managers to make informed decisions that enable future processes of change and any subsequent projects to be more effective.

​

Types of Evaluation Methods

​

Evaluation can be defined as the structured interpretation and giving of meaning to predicted or actual impacts of proposals or results. It looks at original objectives and at what is either predicted or what was accomplished and how it was accomplished.

​

There are a range of evaluation methods that can be employed in a project, but no one single method fits all types of projects. The evaluation approach chosen will depend on the specific project in question and also the purpose of the evaluation. Not all evaluations serve the same purpose. For example, some evaluations serve a monitoring function and others may focus solely on measuring project outcomes or evaluating findings. Furthermore, the needs of the client, its partners and stakeholders, including any funding agency will also influence the nature and scope of the evaluation undertaken. What is clear though, is the need to engage and consult with relevant stakeholders at the outset to agree on the most suitable approach and then jointly design and plan the evaluation process.

​

Evaluation can take place in the period leading up to the design and development of a project as well as during its implementation, with the intention of providing information to improve the design and effectiveness of a project. This is known as formative evaluation. Formative evaluation often lends itself to qualitative methods of inquiry. Formative evaluation is especially important in community engagement and behaviour change projects. Such interventions are often complex and therefore require careful monitoring of processes in order to respond to emergent properties and any unexpected outcomes.

​

Evaluation can also take place during and following project implementation and be used to provide information on the effectiveness or impact of a project and draw lessons for the future. This is known as summative evaluation. Summative evaluation is associated with more objective, quantitative methods. Summative evaluation is also outcome-focused more than process focussed.

​

Monitoring and Evaluation Good Practice

​

We believe it is good practice to make monitoring and evaluation an integral part of the entire project cycle. This is because it is important to know both how a project is working (while it is being implemented) as well as if it was successful (after it has been completed). We therefore recommend a combination of formative and summative evaluation is conducted for most projects, including:

​

• In the period leading up to the design and development of a project (formative).

​

• During its implementation (formative).

​

• In the period following project implementation (summative).

​

We also believe it is important that agreement on the most suitable approach and design of a monitoring and evaluation system to be employed for a specific project should be reached during the project design process. Our approach is to engage and consult with all relevant stakeholders on this matter at the outset of the project design process. This enables a clear plan for monitoring and evaluation to be developed before a project has started. In this way, important decisions can be taken that may influence the design and implementation of that project. This includes what questions about the project need to be answered by the monitoring and evaluation process, what performance indicators will be used and therefore what data is needed to measure performance, how will the data be collected and analysed and how will the results be disseminated.

​

Monitoring and evaluation can serve many purposes and how the results and impacts of any intervention are viewed, analysed and acted on can vary depending on the needs and objectives of those monitoring, evaluating and experiencing change. Therefore, it is crucial that careful consideration is given to these factors when deciding on the approach to adopt. In this light, we recommend that the process of deciding on the detail of the specific monitoring and evaluation system to be employed should include consideration of:

​

• The purpose of any monitoring and evaluation exercise,

​

• What should be monitored and evaluated,

​

• Who should participate in the design and implementation of the monitoring and evaluation exercises.

​

• How and when it should the monitoring and evaluation be carried out.

​

• Any specific stakeholder priorities or requirements regarding monitoring and evaluation.

​

• The availability of resources to establish an effective monitoring and evaluation system.

​

For most of the assignments we are involved in, we recommend a participatory approach is adopted across the entire monitoring and evaluation process, including the design of the monitoring and evaluation exercises to be undertaken. This approach encourages the participation of stakeholders who are directly involved in or affected by the project’s activities in the process of determining the questions to be asked, the types of information to be obtained, selecting the indicators to be used to measure success and in the collection of information and the evaluation of findings. We believe this can make monitoring and evaluation more effective by including a wider range of stakeholders at every stage of the process. This approach can be applied across a range of purposes and contexts at the local or community level, including in the fields of economic and community development, health, education and urban settings.

​

We believe greater emphasis should be placed on achieving results and objectives beyond the financial reporting of traditional forms of evaluation often prescribed by the needs of funding agencies and policymakers.

​

We have seen the emphasis moving away from externally controlled ‘data-seeking’ evaluations towards recognition of locally-relevant or stakeholder-based processes for gathering, analysing, and using information. This approach enables people to reflect on past experience, examine present realities, revisit objectives and define future strategies, by recognising different needs of stakeholders and negotiating their diverse claims and interests.

​

Benefits of Evaluation on the Change Process

 

Adopting the approach outlined above for monitoring and evaluating a project has a number of advantages and benefits for the client, the project, the intended recipients and for the design and implementation of future projects. For example, this approach can:

​

• Provide a better understanding and greater clarity regarding the need for a project.

​

• Enable adjustments and modifications to be made to a project’s design as it is being rolled out, thus improving the likelihood of achieving the desired outcomes at the end of the process. For example:

​

• It can allow early improvements and refinements to be made to the change process while a project proposal is going through the initiation and developmental stages and before it can be implemented fully.

  ​

• It can avoid embarking on a project that does not meet a real need, or one that may be constrained by external factors that cannot be controlled.

​

• Help to ensure that a project’s activities are being delivered efficiently and effectively.

​

• Provide a better understanding of the process of change as well as finding out what works, what does not and why. This allows an opportunity to gather the knowledge to learn and improve future project designs and implementation.

​

• Determine how effective a project has been, in terms of whether the project’s activities and the resultant outcomes have achieved the project’s stated purpose, goals and objectives.

​

• Consider both the positive and intended impacts, as well as the negative and unintended impacts of an intervention.

​

• Determine the extent to which the activities undertaken by a project - or the external environment - has given rise to the impacts recorded (which can be important for policy makers, funders and practitioners to know whether to expand, modify, or eliminate specific interventions).

​

• Determine how effective the strategies and procedures involved in the implementation of a project have been in helping to direct and manage the change process.

​

• Verify whether predetermined targets have been met - including those set by people responsible for implementing the project as well as the project funders.

​

• Compare the impact of different projects and make results-based decisions on future spending allocations.

​

• Decide whether to continue with the program or to end it. It is used to determine whether a project is sustainable and whether it should be extended to other locations.

​

How We Can Help You: Monitoring and Evaluation

​

We design bespoke approaches to performance monitoring and evaluation to meet the specific needs of each project, whether small or large scale. We support projects being run by the public sector, businesses and community organisations and groups.

​

We design and conduct performance monitoring exercises to enable on-going analysis of the progress of a project towards achieving its planned outcomes, deliverables, milestones and financial targets.

​

We design and implement project evaluation strategies and plans to determine whether a project’s outcomes have succeeded in bringing about the changes and benefits as defined by the original project purpose, goals and objectives.

We are also able to assess how well the process of change was managed, including what did and did not work and why, as well as help consolidate what has been learnt from the change to help improve the change process in the future.

​

We are able to employ a range of monitoring and evaluation methods as no one single method fits all types of projects. The evaluation approach chosen will depend on the specific project in question and also the purpose of the process. In choosing the most appropriate methodology, we will take account of the needs of the client, its partners and stakeholders, including any funding agency. The needs of all stakeholders may influence the nature and scope of the evaluation undertaken.

​

One of our key roles in this approach is to facilitate the inclusion of a wider number of stakeholders on the premise that this will result in a number of ideas and perspectives. We also take a leading role in helping to establish and design the monitoring and evaluation system to be employed. We will also lead on the actual facilitation of the monitoring and evaluation process and employ both qualitative and quantitative methods for data gathering and analysis and learning from findings.