Monitoring and Evaluation

1 www.mgtwell.com | [email protected] | +93 7795 01150

2 www.mgtwell.com | [email protected] | +93 7795 01150
 Getting to Know Each Other

“  Full Name

 Current Position

 Work Experience

 Expectations

3 www.mgtwell.com | [email protected] | +93 7795 01150

 Training Contents
Session 1: Understand M&E
Session 2: Data Collection and Management
Session 3: Logical Farmwork

4 www.mgtwell.com | [email protected] | +93 7795 01150

 Training Objectives
After the course, participants will be able to:
• Understand Monitoring and Evaluation

• Collect and analyze the data

• Make M&E plan

• Develop Logical framework for their project

5 www.mgtwell.com | [email protected] | +93 7795 01150

 Training Regulations

Phone off No smoking Active listening

Raising ideas Group activities Punctuality

6 www.mgtwell.com | [email protected] | +93 7795 01150
 Learning Methods

Lecture Role-play Group discussion

Games Case-study

7 www.mgtwell.com | [email protected] | +93 7795 01150

 Pre-test Assessment

Objective: (Formative Evaluation)

A quick & easy way to identify the background knowledge

of participants about the topic prior the training.

8 www.mgtwell.com | [email protected] | +93 7795 01150

 Monitoring and Evaluation

Plan
‫الن‬000‫پ‬
‫ذاری‬000‫گ‬

M&E System.‫ظام‬..‫ن‬ Objective

‫ظارتو ارزیابی‬..‫ن‬ ‫ف‬.‫هدا‬.‫ا‬

To reach Paris‫رسیدن‬
‫اریس‬000‫ پ‬0‫ه‬000‫ب‬

9 www.mgtwell.com | [email protected] | +93 7795 01150

 Monitoring and Evaluation
• Monitoring and Evaluation (M&E) is a crucial
process for assessing and improving the
effectiveness of programs, projects, or
interventions.
• Understanding M&E principles and techniques can
help you guide organizations and individuals in
designing, implementing, and measuring the
impact of their initiatives. Here's some useful
information on monitoring and evaluation:

10 www.mgtwell.com | [email protected] | +93 7795 01150

 Monitoring and Evaluation
• Definition of M&E: Monitoring involves the continuous
tracking of project activities and outputs to ensure they are
on track and identifying any deviations.
• Evaluation, on the other hand, is a periodic assessment of
the project's outcomes and impacts, examining the overall
success and lessons learned.
• Purpose of M&E: M&E helps organizations answer critical
questions like:
• Is the project or program achieving its intended outcomes?
• Are the activities being implemented as planned?
• Are there any challenges or risks that need to be addressed?
• What changes are needed to improve performance and results?
11 www.mgtwell.com | [email protected] | +93 7795 01150
 Monitoring and Evaluation
• Indicators and Targets: Key performance indicators (KPIs) and
targets should be established during the planning phase.
Indicators are specific, measurable variables that demonstrate
progress towards the project's objectives. Targets are the
desired values or levels to be achieved within a set timeframe.
• Data Collection Methods: M&E relies on collecting both
qualitative and quantitative data. Common data collection
methods include surveys, interviews, focus group discussions,
observations, and document reviews.
• Data Analysis: Collected data needs to be analyzed to make
sense of the findings. This can involve statistical analysis,
thematic coding of qualitative data, or comparing results against
predetermined targets.
12 www.mgtwell.com | [email protected] | +93 7795 01150
 Monitoring and Evaluation
• Data Visualization: Presenting M&E data in a visually engaging
way helps stakeholders understand and interpret the findings
better. Charts, graphs, and infographics are useful tools for this
purpose.
• Learning and Adaptation: M&E is not just about reporting; it's
about learning and adaptation. Encourage organizations to use
the findings to make informed decisions, adjust strategies, and
improve their programs continually.
• Challenges and Ethical Considerations: M&E may face
challenges such as limited resources, data quality issues, or
resistance to evaluation findings. Additionally, ethical
considerations, like ensuring participant confidentiality and
informed consent, should always be observed.
13 www.mgtwell.com | [email protected] | +93 7795 01150
 Monitoring and Evaluation
• M&E Framework: Developing a comprehensive
M&E framework early on helps ensure that all
aspects of monitoring and evaluation are well-
organized and aligned with the project's objectives.
• Sustainability: M&E efforts should not end with
project completion. Encourage organizations to
institutionalize M&E practices and build a culture of
learning and evidence-based decision-making.

14 www.mgtwell.com | [email protected] | +93 7795 01150

 Monitoring

Monitoring is a systematic and continuous process of:

Data Collection
Data Analysis and Using information
Report to Decision
Makers and
Management Control

15 www.mgtwell.com | [email protected] | +93 7795 01150

 Evaluation

Evaluation in its broadest sense means “to assess or judge the worth or value of something”.

Relevance

Impact

Efficiency

Effectiveness

Sustainability

16 www.mgtwell.com | [email protected] | +93 7795 01150

 importance of project cycle in monitoring and evaluation

• The project life cycle is essential for NGO/CSOs as it

provides a structured framework for managing a
project from beginning to end.
• It includes stages of design, implementation,
monitoring and evaluation, which helps focus
efforts on monitoring performance and evaluating
long-term results.

17 www.mgtwell.com | [email protected] | +93 7795 01150

 Result Based M& E approach

• M&E is part of a results-based management system that

focuses on outcomes and impact, using three
approaches to contribute to system innovation.
• 1.result-oriented Approach
• a result-oriented approach refers to the systematic and
purposeful focus on achieving desired outcomes and
impacts of a project, program, or intervention.
• It places a strong emphasis on measuring and assessing
the actual results and effects of activities and initiatives,
rather than merely tracking inputs and outputs.

18 www.mgtwell.com | [email protected] | +93 7795 01150

 Result Based M& E approach
• Reflexive E Approach
• Reflexive Monitoring is an innovative monitoring
and evaluation method that aligns daily activities
with long-term ambitions and project impact.

19 www.mgtwell.com | [email protected] | +93 7795 01150

 Participatory Monitoring
approach
• The Participatory Monitoring (PM) approach is a
methodology in the field of Monitoring and
Evaluation (M&E) that emphasizes the active
involvement of stakeholders, including beneficiaries
and local communities, throughout the entire M&E
process.

20 www.mgtwell.com | [email protected] | +93 7795 01150

 Theory of change, result chain and logical
framework

• A theory of change is a conceptual framework that

describes how and why a project is expected to
lead to desired results.

21 www.mgtwell.com | [email protected] | +93 7795 01150

 Progress & results monitoring or implementation &
outcome monitoring

• Progress Results monitoring is a type of M&E that

monitors the implementation of agreed outputs,
including compliance, financial, process, and
beneficiary monitoring.

22 www.mgtwell.com | [email protected] | +93 7795 01150

 Monitoring Vs Evaluation

Monitoring Evaluation
When is it done? Continuous At fixed points
What information is Directly available information More detailed information; may
collected? about outputs be harder to get

With what purpose? To check that activities are To see whether the goals and
implemented as planned objectives are being reached

Who does it? Program staff as part of Internal, or external team with
their day to day work specialist knowledge, assisted by
program staff

How is the result To improve quality of To judge the impact on the target
used? implementation and adjust population, adjust objectives;
planning. As input to decide about the future of the
evaluation. program.

23 www.mgtwell.com | [email protected] | +93 7795 01150

 Why monitor and evaluate

• Ensure accountability

• Aid in decision making

• Tools for learning and reflection

• Improve project design

• Provide evidence for advocacy and obtaining

support
24 www.mgtwell.com | [email protected] | +93 7795 01150 24
 Participatory monitoring and
evaluation cycle

Take action Decide who

participates

Share results
Establish
goals

Analyze
Develop
results
indicators
Gather
information

25 www.mgtwell.com | [email protected] | +93 7795 01150 25

 Types of monitoring activities

• Institutional monitoring – internal monitoring of financial,

physical and organizational issues affecting the project
• Context monitoring – tracking the context in which the
project is operating as it affects critical assumptions and risks to the
project
• Results monitoring – tracking project effects/outcomes and
impacts, much is handled during evaluation
• Objectives monitoring – tracking project objectives and
strategies in relation to continuing relevance to the target population
and its changing needs

26 www.mgtwell.com | [email protected] | +93 7795 01150 26

 Types of Evaluations

• Formative Evaluation

• Formative evaluations are evaluations

intended to improve performance, and are
most often conducted during the
implementation phase of projects or
programs.
• Formative evaluations may also be
conducted for other reasons, such as
compliance and legal requirements.
27 www.mgtwell.com | [email protected] | +93 7795 01150
 Types of Evaluations
• Summative Evaluation
• Summative evaluations are studies conducted at the end of
an project (or phase of that intervention) to determine the
extent to which anticipated outcomes were produced.

• Summative evaluation is intended to provide information

about the worth and impact of a program.

• Related terms: Ex-post Evaluation, Post Implementation Reviews

(PIRs) and Impact Assessment.

28 www.mgtwell.com | [email protected] | +93 7795 01150

 Types of Evaluations
• Prospective Evaluation
• Prospective evaluation, also known as ex-ante
evaluation or forward-looking evaluation, is a type
of evaluation that takes place before a project or
program is implemented.
• It is somewhat similar to an evaluability assessment. An
evaluability assessment answers the questions “Is this
program or project worth evaluating?” and “Will the
gains be worth the efforts/resources expended?”

• Related terms: Ex-ante (before the fact)

29 www.mgtwell.com | [email protected] | +93 7795 01150

 Main evaluation points during the
project cycle
• Baseline study – done at the beginning of project intervention to
establish benchmarks for select indicators
• Annual review – assessment of outcomes and project strategies over
successive one year periods
• Mid-term evaluation – usually external assessment focusing on
project performance, organizational capacity and mid-term corrections to
improve achievement in the remaining periods
• Final evaluation – external or internal assessment of
effects/outcomes and impacts including cost effectiveness, done just before
the project ends
• Ex-post evaluation – an external and in-depth study of the impact of
a project on the target population, usually conducted 5-10 years after

30 www.mgtwell.com | [email protected] | +93 7795 01150 30

 Four main phases of M&E
• Planning M&E – deciding what info to track,
responsibilities, when to collect info, what methods to use for
data collection and who should receive what information
• Data gathering – use of tools for collecting data at
designated times during the project period
• Analysis and validation – organizing data and
interpreting them into useful information for various users
• Sharing and utilization of findings – transforming
information into various forms for wider dissemination and use
of different stakeholders

31 www.mgtwell.com | [email protected] | +93 7795 01150 31

 Ethical Procedures in M&E

• Informed consent
• Voluntary participation
• Do no harm
• Confidentiality
• Anonymity

32 www.mgtwell.com | [email protected] | +93 7795 01150

 Project stages and information needs
What are the problems?
Before
project What are the resources?
What are the unmet needs?
Project
start-up What is the current situation?

Implementation Is the project proceeding according to plan?

Mid-term Are the project strategies working?

End of project What effects did the project have?

What impact did the project have on the lives of the

After project people it was designed to affect?

33 www.mgtwell.com | [email protected] | +93 7795 01150 33

 Steps in developing a monitoring and
evaluation plan
• State the project’s goal’s and objectives
• List indicators for each goal, specify the data to
be collected
• State the data collection methods, tools,
sources, data gatherers and dates
• Explain how the data will be analyzed
• Describe the method to disseminate
information to key stakeholders

34 www.mgtwell.com | [email protected] | +93 7795 01150 34

 Hierarchy of project goals and
objectives
• Goals – what the project intends to contribute in the long term
• Objectives – what response the project intends to achieve among
the target population groups
• Outputs – what the project intends to achieve in the short term as a
result of the project activities
• Activities – what the project staff and target population are going to
do
• Inputs – what resources are necessary for performing the project
activities

35 www.mgtwell.com | [email protected] | +93 7795 01150 35

 Indicators

Qualitative or quantitative criteria used to check

whether proposed changes have occurred
• Measure of project success or progress
• Not same as targets
• Not generally presented as
numerical targets in themselves

36 www.mgtwell.com | [email protected] | +93 7795 01150 36

 Five main types of indicators
corresponding to the hierarchy of
objective and results
Objective Indicato Description of Example
r type indicator type
Goal Impact Sustainable change in the Household security levels shown
conditions of the basic by measures of health, nutrition,
problem identified education, economic security
Objective Effect/ Target population’s % of households in the area
outcomes responses to project using energy saving stoves
outputs
Output Output Project products, direct Number of health workers
outcome of project trained
activities

Activities Process Project activities or Number of training events held

processes Number of household visits
Inputs Input Resources that go into the Number of health kits distributed
project Number of staff supported by the
project

37 www.mgtwell.com | [email protected] | +93 7795 01150 37

 Criteria for selecting good indicators

• Relevant Comprehensible
• Technically viable Valid
• Reliable
Sensitive
• Usable
Cost-effective
• Participatory
Timely
Ethical

38 www.mgtwell.com | [email protected] | +93 7795 01150 38

 Common types of indicators
Indicator types What they show Examples
Indicators of Whether something exists and if it is Number of school for girls in each
availability available of the districts covered

Indicators of How relevant or appropriate something New seeds give better yields
relevance is
Indicators of Whether what exists is actually within Number of women applying for
accessibility reach of those who need it loans for income generating
project

Indicators of What extent something that has been Number of couples availing family
utilization made available is being used planning services in the clinic

Indicators of What proportion of those who need % of school age girls who go to
coverage something are receiving it school

Indicators of quality Quality or standard of something Carpets follow international

design
Indicators of effort How much and what is being invested The amount of time to construct a
to achieve objectives 6-room school

Indicators of Whether resources and activities are % of reduction in wastage of

efficiency being put to best use materials used

Indicators of impact What you are doing show any difference Increase household income by
amount of Afs per year

39 www.mgtwell.com | [email protected] | +93 7795 01150 39

 Basis for selecting data collection
methods

• Existing or new data – what secondary info is available,

disaggregated and accessible; how about quality?
• Qualitative or quantitative – numerical or description of
attitudes, beliefs and perceptions
• Participatory or non participatory – high perception of
ownership for participatory process
• Verbal or less verbal – low literacy or mixed language
target population or desired information is not easily expressed in
words or numbers

40 www.mgtwell.com | [email protected] | +93 7795 01150 40

 Methods for data collection

• Observation
• Interviews
• Discussions
• Surveys
• Project visits
• Other
participatory
tools: PRA

41 www.mgtwell.com | [email protected] | +93 7795 01150 41

 What is Data Analysis?

• The process of understanding and explaining what

findings actually mean. Turning raw data into useful
information
• Provide answers to questions being asked at a program
site or research questions being studied
• The greatest amount and best quality data mean
nothing if not properly analysed, or, if not analysed at
all

42 www.mgtwell.com | [email protected] | +93 7795 01150

 What is Data Analysis?

Analysis is looking at the data in light of the questions

you need to answer

Question Data Analysis Answers

How would you analyze data to determine, “Is my

program meeting it’s objectives?”
43 www.mgtwell.com | [email protected] | +93 7795 01150
 Steps in analyzing data

1. Go back to the original objectives

of the needs assessment

2. Remember to think of facts and

their interpretation

3. Follow a systematic and logical

path in the analysis

4. Discuss the findings and

emerging analysis

5. Consider any limitations to

believability

6. Make specific recommendations

44 www.mgtwell.com | [email protected] | +93 7795 01150 44

 Logical steps in data analysis

Organize raw data

Prepare description

Generate interpretation

45 www.mgtwell.com | [email protected] | +93 7795 01150 45

 Types of data analysis

• Quantitative techniques – expressed in numbers, some

descriptive statistics can be done with a hand calculator (tallies,
frequency, averages, proportions, percentage)
• Qualitative techniques – use direct content from
respondents such as quotes (tables, flow charts would help in
seeing trends and patterns in the data generated). This requires
good summarizing skills and insight capabilities
• Combining these techniques – can be used for cross
checking interpretation

46 www.mgtwell.com | [email protected] | +93 7795 01150 46

 Is Our Program on Track?

• Analysis: Compare program targets and actual

program performance to learn how far you are
from target
• Interpretation: Why you have or have not
achieved the target and what this means for your
program
• May require more information

47 www.mgtwell.com | [email protected] | +93 7795 01150

 Examples of Analysis

Compare actual performance against targets

Indicator Progress (6/12/13) Target (1/30/14)
Number of persons trained on 15 100
case management

Comparing current performance to prior year

Indicator 2011 2012
No. of LLIN distributed 50,000 167,000

Compare performance between sites or groups

Indicator District A District B
Number of fever cases tested for 3,500 8,000
malaria by clinics

48 www.mgtwell.com | [email protected] | +93 7795 01150

 Statistical Measures

• Measure of central tendency

– Mean
– Median
– Mode

• Measure of variation
– Range
– Variance and standard deviation
– Interquartile range
– Proportion, Percentage
• Ratio, Rate

49 www.mgtwell.com | [email protected] | +93 7795 01150

 Mean

Average number of confirmed malaria cases

per month
Month Cases 2008 Total number of cases
Jan 30
Sum of the values Feb 45
∑ yi =1,180
divided by the number Mar 38

of cases. Also called April 41

Number of observations
average May 37

∑ yi Jun
Jul
40
70
n= 12
̄y =
n Aug
Sep
270
280
Mean number of cases

Very sensitive to variation

Oct 200 1,180
Nov 100 ̄y = =98 . 2
Dec 29
12
50 www.mgtwell.com | [email protected] | +93 7795 01150
 Median

Median number of confirmed malaria cases

• Represents the middle Month Cases

2008
Cases
2009
Median for 2008
of the ordered sample
41+45
data Dec 29 24
median=
2
=43

• For odd sample size, Jan

May
30
37
29
32
the median is the Mar 38 35
middle value Jun
April
40
41
39
39
Median for 2009
• For even, the median is Feb 45 42
Jul 70 65
the midpoint/mean of Nov 100 80 m edian= 39
the two middle values Oct 200 150
Aug 270 200
Not sensitive to variation Sep 280 -

51 www.mgtwell.com | [email protected] | +93 7795 01150

 Mode

Mode number of confirmed malaria cases

Month Cases Cases

2008 2009
• Value that occurs most Mode for 2008

frequently mod e= none

Dec 29 24
Jan 30 29
May 37 32
• It is the least useful (and Mar 38 35
least used) of the three Jun 40 39
Mode for 2009
April 41 39
measures of central Feb 45 42
tendency Jul 70 65
Nov 100 80 mode = 39
Oct 200 150
Aug 270 200
Sep 280 -

52 www.mgtwell.com | [email protected] | +93 7795 01150

 Practice Calculations

• What is the mode, mean – Would you use Mean or

and median parasitemia Median?
for the following set of – Answer: Median
observations? – Use Median when you
1.5, 1.8, 2.5, 4.1, 8.3, 1.2, have a large variation
between high and low
1.9, 0.6
numbers
• Answers: – Use Mean when there is
– Mean = 2.74 not a huge variation
– Median = 1.85 between the values
– Mode=none
53 www.mgtwell.com | [email protected] | +93 7795 01150
 Ratio

• Comparison of two numbers

• Expressed as:
– a to b, a per b, a:b
– 2 household members per (one) mosquito net, a
ratio of 3:1
• All individuals included in the numerator are not
necessarily included in the denominator
54 www.mgtwell.com | [email protected] | +93 7795 01150
 Proportion

• A ratio in which all individuals in the numerator are

also in the denominator

• Example: If a clinic has 12 female clients and 8 males

clients, then the proportion of male clients is 8/20 or
2/5 FFFF
MMMM
FFFF
MMMM
FFFF
55 www.mgtwell.com | [email protected] | +93 7795 01150
 Percentage

• A way to express a proportion

• Proportion multiplied by 100
• Example: Males comprise 2/5 of the
clients or, 40% of the clients are male
(0.40 x 100)

Important to know: What is the whole? An

orange? An apple? All clients? All clients on
with a fever?
56 www.mgtwell.com | [email protected] | +93 7795 01150
 Why do we want to know the
percentage?

• Helps us standardize so that we are able to

compare data across facilities, regions,
countries
• Better conceptualize what needs to be done
– Percentage helps us to track progress on our
targets

57 www.mgtwell.com | [email protected] | +93 7795 01150

 Rate

(Under five mortality rate)

Probability of Dying Under Age Five per
1,000 Live Births
• A quantity measured with
respect to another Nation Under five mortality
rate per 1,000 live
measured quantity births in 2008

France 4
• Number of cases that occur Ghana 76
over a given time period Sierra Leone 194
divided by population at risk Afghanistan 257
in the same time period
Source: UNICEF: Statistics and Monitoring by Country

58 www.mgtwell.com | [email protected] | +93 7795 01150

 Annual Parasite Incidence (API)

Number of microscopically confirmed malaria cases

detected during one year per unit population

Confirmed malaria cases during 1 year

API X 1000
Population under surveillance

59 www.mgtwell.com | [email protected] | +93 7795 01150

 Most Common Software

• Microsoft Access
• Microsoft Excel
• Epi-Info
• SPSS
• Stata
• SAS

60 www.mgtwell.com | [email protected] | +93 7795 01150

 The M&E matrix should include

• Indicators
Type of activity
• Sources of information
Frequency
• Method for data collection
Application
• Method for data analysis
Circulations

61 www.mgtwell.com | [email protected] | +93 7795 01150 61

 Outline of an M&E plan

• Table of contents (optional)

• Executive summary (500 words to half-page)
• Project background (description of context, objectives,
strategies, conceptual model, current status)
• M&E background (description of the process of designing,
operational terminology)
• M&E plan (purpose, M&E matrix, project risks and assumptions,
responsibilities for M&E, feedback mechanisms)
• Useful annexes (forms, Gantt chart, references)

62 www.mgtwell.com | [email protected] | +93 7795 01150 62

 Steps in engaging professional
evaluators

• Develop terms of reference for an evaluation

• Agree on services required and responsibilities

• External evaluator and project team should work

well together

63 www.mgtwell.com | [email protected] | +93 7795 01150 63

 Important attitudes when conducting an
evaluation

• Evaluation is not a fault finding exercise

• Treat it as a process that facilitate critical thinking
and reflection
• Link the process to the project objectives and
purpose of the evaluation
• Make the whole process a product of teamwork

64 www.mgtwell.com | [email protected] | +93 7795 01150 64

 Important behaviors to observe

• Be a facilitator
• Don’t act like a ‘Boss’
• Don’t act line a ‘Spy’
• Listen very well to what people are saying and not
saying
• Do not analyze information in isolation of the big
picture
• Minimize your biases
• Refer to the TOR
• Look for empowerment and achievement

65 www.mgtwell.com | [email protected] | +93 7795 01150 65

 Guidelines for presenting information

• Know your audience (info users)

• Know what they need to know and why
• Know when the info is needed (timing, frequency)
• Relate the info presented to the anticipated
applications (uses)
• Choose presentation methods that fit the
audience
• Choose a variety of ways to present
information
66 www.mgtwell.com | [email protected] | +93 7795 01150 66
 Uses of M&E results

• To assess project impact

• To improve project planning and
implementation
• To strengthen organizational capacities and
institutional learning
• To build shared meaning
• To demonstrate accountability and influence
policy making or strengthen advocacy

67 www.mgtwell.com | [email protected] | +93 7795 01150 67

 Monitoring and Auditing
•Monitoring and auditing will be
undertaken to determine the impact as
a consequence of the rehabilitation, and
maintenance of the DABS work. General
monitoring and auditing will be
conducted daily throughout the
rehabilitation stage and monthly during
the operation and maintenance phase.

68 www.mgtwell.com | [email protected] | +93 7795 01150

 • Routine monitoring and reporting will be undertaken by
the DABS safeguard team on a regular basis and by the
contractor the E&S and H&S Specialist on a daily basis.
DABS will develop an Environmental, Social and Health
and Safety Auditing Schedule and undertake audits in
accordance with the schedule. In the first year the Audit
is carried out every 4 months, after the first year every 6
months.

69 www.mgtwell.com | [email protected] | +93 7795 01150

 DABS Project ESMP Reporting Tables
Item of ESMP Duration Estimated costs

Cost of the trainings: Actual, During and after US$ 1000. 00

1-Training in the project
Operation/Maintenance implementation
for xxx Project
2- First Aid and safety Actual, Before project US$ 500 . 00
appliances and training implementation time
for XXX

3- Environmental, Social Actual, During the project US$ 300 . 00

and Health and Safety implementation
Officer on site

4- Site restoration and After subproject US$ 1800 .00

replantation and completion.
landscape improvement

70 www.mgtwell.com | [email protected] | +93 7795 01150

 Review, Questions

71 www.mgtwell.com | [email protected] | +93 7795 01150

Thanks