Developing a Monitoring and Evaluation Framework
This document is also available as a downloadable Scribd publication.
A monitoring and evaluation (M&E) framework is the holistic approach that can address your programme, needs, monitor programme processes and outputs and evaluate goals and objectives. It should encompass the programme planning process right down to the documentation and dissemination plan.
An M&E framework should always be developed before a programme has been initiated.
However, despite the pleads for M&E to built in, or rather mainstreamed in all operations, rapidly constructing an M&E framework during a programme rather than before, is quite common (especially among smaller and capacity restricted organizations). Sometimes, this afterthought approach can work, for example in the case of programmes with simple goals – consider a basic capacity building programme with the sole aim of building capacity in certain skills, a simple evaluation of knowledge following the programme at timely intervals can allow you to assess impact. However if your programme seeks to utilize this capacity building for a social/cultural/economic/structural change (which most of us do!) then this afterthought approach is always ill advised.
Although creating a detailed M&E framework for larger programmes is time consuming, the process will be worth the time invested. Not only does an M&E framework help in monitoring and evaluating the programme, it is a detailed planning process that will aid you in strategically thinking through every detail of the programme, from resources required to how the results will be disseminated. It is a key investment that promotes implementation and will ultimately save time and stress.
Although, there are a number of approaches to creating a M&E framework. It is my personal preference to conduct this activity in six steps. For those organizations who are limited in time and capacity and if the programme is not extremely detailed, I suggest that you prioritize Step 1, 2 and 3.
Six Steps to Creating an M&E Framework
Step One – Participatory Programme Logic Planning
This is an approach to programme planning and design. Participatory programme logic planning refers to the process of examining what your desired goal is and strategically thinking through proposed activities to ensure that you maximize your ability to attain this goal. It is recommended that this process is participatory and that a whole range of relevant expertise’s, partners and even the people whose life you wish to positively impact should be involved. Simply because a whole range of stakeholder are involved, it allows you to anticipate risks and acknowledge assumptions you are making that will lead to the success of a programme. It also promotes the examination of unexpected outcomes.
Program logic planning can be represented as vertical logic that will highlight a cause and effect relationship between your programme activities.
Like this: Activities – Outputs – Outcomes – Objectives- Goal. Or as a diagram like the causal pathway below.
This is a very important step – all inputs must be enough to produce activities and these activities must eventually produce the goal. Its logic is based on cause and effect or the ‘theory of change’ but you must be careful to think of challenges during this exercise. Overall the participatory programme logic planning will:
- Help you gain a more fully shared understanding of how they believe change is likely to occur your operational context
- Identify gaps in the initial design of the programme and reach consensus to overcome these challenges
- Form the conceptual basis for the M&E framework
Step Two – Filling in the Logical Framework
Following the programme logic planning you are in a position to fill in the logical framework. As your activities, outputs, outcomes, objectives and goal (s) are already established, you must begin to fill in the full logical framework.
a) Insert all inputs into the log frame
This could include various technical support, human resources, financial resources, equipment etc
b) Establish indicators for all outputs, outcomes, objectives and goal(s)
Indicators enable us to track progress, demonstrate results, and take corrective action to improve service delivery. Establishing indicators is a usually a simple and logical procedure – ask yourself if that output, outcome, objective, goal is achieved then how can we know? It must follow the SMART principal. An example of indicators for a goal is as follows:
Goal: A program has the goal to increase community knowledge of HIV prevention and access to HIV testing treatment and care through targeted health talks
- Number of community members reached
- Improvement in community members knowledge on HIV
- Improvement in number of people testing
- Improvement in the number of people receiving treatment
- Increase in condom use
- Reduction in HIV transmission
The number of indicators per output, outcome, objective, and goal depends on the strength of the indicator to capture the achievement. Indictors must also be numerated when applicable. Please do that when we talk about any of the below type of indicators: Increase, Decrease, Improvement, Change, Etc. We can only measure these if we have a BASELINE to compare it with. A baseline is information on whatever we wish to impact prior to our program intervention.
c) Ascertain all indicators have a means of verification (MoV)
The indicators that you created are only useful when supported with sound data collection. For every indicator you create you must be able to verify it. That is, you must be able to collect data to prove that your indicator is true.
Indicator: Decrease in the number of TB patients lost to follow up
MOV: National TB registers
Indicator: Increase in TB/HIV knowledge among clinic staff
MOV: TB/HIV knowledge questionnaire
In the end, MoV’s are the data collection tools you will use to monitor progress and evaluate impact. Some may already be in existence such as national registers; others will have to be created. A list of all the data collection tools that are already available and those that need to be designed should be prepared. Simple design is always the best, do not collect too much data that is irrelevant to your programme and above all ensure that they collect the desired indicator (s)!
d) Insert a time line for the collection of each indicators and persons responsible.
Some indicators may be collected daily and others may only be collected annually. A time line should be devised for the collection of all indicators along with an allocation of persons responsible. Timelines will support in project planning and consistent data collection.
e) Based on your programme logic discussion, consider all assumptions to your programme’s success.
As we all know, despite careful programme logic planning this does not guarantee achieving the programme goals. For example: supplying ARVs to a rural clinic to support treatment for HIV may not translate to these ARVs being utilized. You are making the assumption that:
- Community members are open to HIV testing and blood draws
- Community members found positive is willing to use ARVs
- Community members can avail of adequate nutrition to help take the ARVs with
- There is no stigma or non disclosure in the community
- There are no misconceptions and myths in the community
- The community members further from the clinic can avail of transport…
You must note all your assumptions to your projects success and consider methods to mitigate these, should they arise.
Following the creation of the logical framework there are a number of other steps to complete for the development of a full and detailed M&E framework.
Step 3 Data Tool Collection Guide
In the past, I have found it extremely useful to create a data collection tool guide. This is a guide on how to use all the data collections tools stipulated in the logical framework. It should have the following content on each tool:
- What the tool is? (approach/method/design)
- How to use the tool?
- When to use the tool?
- Persons responsible for using the tool
- What indicators the tool collects
I have found it quite useful, to have a data collection tool guide in circumstances where people who will be collecting the data have little experience with M&E. A brief training should be conducted on the data collection tool guide.
Step 4 Reviewing Revising and Reporting Plan
The reviewing revising and reporting (RRR) plan is an integral part of a monitoring and evaluation framework. The purpose of the reviewing, revising and reporting plan is to support the implementation of the M&E framework. The RRR plan is usually a 12 month plan. RRR plans provide an internal RRR structure, however the reviewing and revising component should actively work in conjunction with external reporting requirements.
The objectives of an RRR plan are as follows:
- To monitor progress made on programme activities and indicators
- To Identify and correct programmatic /data collection challenges early
- To promote programmatic structure and internal reporting as a routine function of the implementing team
Reviewing is a process by which the implementing teams’ activities, indicators and data collection methods are consistently monitored. Its purpose is to focus on program’s overall achievements, challenges and emerging issues, based on information obtained through the M&E framework/processes.
Revising is the action the implementing team takes following a reviewing process. This could mean redesigning an activity area that is failing to meet indicator requirements, changing a data collection tool to make it more user friendly or focusing more specifically on a particularly successful activity area.
Reporting is a routine and focused process by which the implementing team collectively discusses progress towards established programmatic indicators.
The RRR plan combines textual and visuals descriptions of how the RRR component of the M&E framework will be implemented. This could include monthly indicator meetings, quarterly revising processes and interim and annual reporting. It should also describe how information flow is conducted and managed.
Step 5 Documentation and Dissemination Plan
Information provided by a strong M&E framework is invaluable. Organizations should always try to document: good practices, success stories and case studies. Information from a well monitored and evaluated programme will facilitate the sharing of ideas for action among other organizations working in a similar focus area. A good process of documentation will provide information on what people are doing and what’s working, what communities have learned from their experience and how it made a difference. They will provide inspiration and show what’s possible. Plans for disseminating the M&E findings (to journals, online forums, partners etc) including the documentation should form a critical part of the overall M&E activities.
Step 6 M&E Plan
An M&E plan is the summary of the M&E framework. Very simply, it is a description of how in practice, the logical frame, RRR plan and documentation and dissemination plan work will work to effectively monitor and evaluate the programme. It should be approx 3-4 pages long.