Population Health for Nurses · Implementation and Evaluation Considerations
Evaluation Strategies
On this page 9 sections
In 30 seconds
Evaluation is the systematic collection, analysis, and use of information to judge how well a program is working and to guide decisions about it. It answers questions like: Did we deliver what we planned? Did participants change? Did health improve? Was it worth the cost? Evaluation is not a verdict delivered at the end of a program; it is a continuous practice that runs alongside implementation, feeding information back so the program can improve in real time.
Different questions require different strategies. Formative evaluation Evaluation during development to shape the program Full entry → informs program development. Process evaluation Evaluation of whether the program was delivered as planned Full entry → asks whether the program was implemented as planned. Outcome evaluation Measurement of short- and medium-term changes in participants Full entry → asks whether participants changed as intended. Impact evaluation Assessment of longer-term effects and their causes Full entry → asks whether the program caused longer-term changes. Choosing the right strategy — and being honest about what a given design can and cannot prove — is the heart of competent evaluation.
Why this matters
- Accountability. Funders, communities, and institutions deserve to know whether programs work and how resources were used.
- Improvement, not just judgment. Good evaluation tells nurses what to fix — which sessions, which outreach, which processes — not merely whether the program "passed."
- Ethics. Continuing a program that does not help — or worse, harms — wastes community resources and erodes trust. Evaluation is how that is caught.
- Local evidence. Clinicians use evidence to choose interventions; evaluation produces the local evidence that an intervention should keep being used here.
- Sustainability. Demonstrated results are the strongest argument for continued funding (see Funding and Sustainability).
- Nursing scope. Population-health nurses often design, run, or interpret evaluations — and must understand what the numbers can and cannot claim before acting on them.
The college version
Core Concepts
Formative evaluation: building the program right
Formative evaluation happens while a program is being developed or refined: needs assessment (what does the community actually need?), pre-testing materials (do people understand the message?), pilot testing (does the session flow work?), and early feedback loops. Its purpose is to shape the program before large resources are committed.
Process evaluation: was it implemented as planned?
Process evaluation measures implementation itself: How many sessions were delivered? How many people attended? Was the curriculum delivered with fidelity (see Facilitators and Barriers to Program Implementation)? Process data answer "did we do what we said we would do?" A program can fail to produce outcomes simply because it was never fully delivered — process evaluation reveals that.
Outcome and impact evaluation: did it work?
Outcome evaluation measures short- and medium-term changes in participants: knowledge, attitudes, skills, behavior, or health indicators. Impact evaluation looks at longer-term effects and tries to attribute them to the program. The key scientific question for both is comparison: how do we know the change didn't happen anyway? Designs include pre/post measurement, comparison groups, and (where feasible) randomization. Real-world programs often cannot randomize, so evaluators must state limits: without a comparison group, you can describe change but cannot confidently claim the program caused it.
Logic models: the roadmap
A Logic model A diagram of inputs → activities → outputs → outcomes → impact Full entry → is a one-page picture of how the program is supposed to work: inputs (resources: staff, money, materials) → activities (what the program does: classes, screenings, referrals) → outputs (direct products: sessions held, people served) → outcomes (short-term changes: knowledge, behavior) → long-term impact (health improvement). Building one with stakeholders forces agreement on the program's theory — and gives evaluation a checklist: if outcomes are missing, the model shows where the chain broke.
Indicators and measures
An Indicator A specific measurable sign of an outcome Full entry → is a specific, measurable sign of progress toward an Outcome A change in participants (knowledge, behavior, health) Full entry →. Good indicators are defined before data collection, tied to the outcome they represent, and measured with instruments that are valid (they measure what they claim) and reliable (they give consistent results). Measures must also be culturally and linguistically appropriate — a questionnaire validated in one language or population is not automatically valid in another.
Quantitative, qualitative, and mixed methods
Numbers (surveys, attendance, records) answer how much; words (interviews, focus groups, open-ended feedback) answer why and how it felt. Mixed methods Combining quantitative and qualitative data Full entry → use both: a survey can show attendance fell, and interviews can explain that participants found the session times impossible. The qualitative story often holds the actionable insight.
Data quality and fairness
Evaluation data can mislead through bias: survey responders may differ from non-responders; self-reported behavior is not observed behavior; measures may work differently across cultural groups. Plan for missing data, collect demographics to check whether the sample matches the intended population, and interpret findings with caution. Privacy and confidentiality rules — what data may be collected, stored, and shared — apply to all evaluation activity and vary by institution and jurisdiction.
Using findings: feedback and continuous quality improvement
Findings should flow back to staff, participants, and community partners and drive program adjustments — the cycle of Continuous quality improvement The cycle of plan, implement, evaluate, adjust Full entry → (CQI): plan, implement, evaluate, adjust, and evaluate again.
How It Works / Step-by-Step Process
- Engage stakeholders. Ask funders, staff, and community members what questions the evaluation must answer.
- Build (or review) the logic model. Agree on inputs, activities, outputs, and intended outcomes.
- Choose the evaluation type. Formative for development, process for delivery, outcome/impact for effects — often more than one.
- Define indicators and measures. Specify data sources, collection methods, and timelines.
- Plan for quality and ethics. Address sampling, missing data, confidentiality, and any review-board requirements.
- Collect and analyze. Track implementation data continuously; analyze outcome data at planned points.
- Interpret honestly. State what the design can and cannot prove; note limitations.
- Report and act. Share findings, make adjustments, and document the cycle.
Common Confusions
| Do not confuse | With | Difference |
|---|---|---|
| Outputs | Outcomes | Sessions held and people served are outputs; changes in knowledge or behavior are outcomes. Counting attendance proves activity, not effect |
| Process evaluation | Outcome evaluation | Process asks "did we do it right?"; outcome asks "did it work?" — both are needed |
| Describing change | Proving cause | Pre/post change can be coincidence, maturation, or outside factors; attribution needs a comparison group |
| Evaluation | Research | Both use rigorous methods, but evaluation serves program decisions; research aims to generate generalizable knowledge — oversight rules differ by institution and purpose |
| Correlation | Causation | Two things changing together does not mean one caused the other |
| More data | Better evaluation | Data must answer the evaluation questions; irrelevant or low-quality data add cost, not insight |
Eli explains
The same idea, in plain words
Explain it like I’m 10
Evaluation is like a teacher checking how the class is learning, not just grading a final exam. Before class, she asks what students already know (formative). During class, she checks whether she actually taught everything she planned (process). After class, she tests whether they learned (outcome). And to be fair, she compares with what would have happened if the class never met. A good teacher uses all of these checks to make the next lesson better — and so does a good health program.
Worked example
A school health team runs an asthma education series for students with asthma and their families. The nurse evaluator builds a logic model with teachers and families: activities are six classroom sessions and two family workshops; outputs are sessions delivered and students attending; outcomes are asthma knowledge, correct use of action plans, and fewer missed school days.
The evaluation uses all four strategies. Formatively, the team pilots the first session and discovers the worksheets are too wordy — they simplify them before the series starts. Process data show attendance held at 80% but the family workshops were poorly attended, so the team moves them to evening hours mid-series. Outcome data show improved knowledge from pre- to post-test. The team wants to claim the program reduced missed school days, but the nurse notes honestly that without a comparison group of similar students, the reduction can be described but not confidently attributed to the program. She recommends a comparison design for the next year — the honest limitation becomes the plan for stronger evidence.
Key takeaways
- Match the strategy to the question: formative (develop), process (implemented as planned?), outcome (did participants change?), impact (did the program cause it?).
- A logic model (inputs → activities → outputs → outcomes → impact) is the roadmap for design and evaluation.
- Process evaluation prevents the classic error: blaming the program when it was never actually delivered.
- Attribution requires comparison; pre/post alone describes change.
- Choose indicators before collecting data; ensure measures are valid, reliable, and culturally appropriate.
- Mixed methods connect the numbers (how much) with the story (why).
- Feed findings back into the program — evaluation exists to improve, not just to judge.
- Respect privacy and confidentiality; data rules vary by institution and jurisdiction.
Check yourself
5 review questions from the chapter. Try each one, then open the answer.
Match each strategy to its question: formative, process, outcome, impact.
Show answer
Formative → "How should we build/refine the program?"; process → "Was it delivered as planned?"; outcome → "Did participants change?"; impact → "Did the program cause the longer-term changes?"
Why is a logic model valuable before data collection begins?
Show answer
It makes the program's theory explicit (inputs → activities → outputs → outcomes), so everyone agrees on what the program is supposed to do, and it provides an evaluation checklist — when outcomes are missing, the model shows where the chain broke.
A program reports "90% of participants attended all sessions" — Output A direct product of activities (sessions held, people served) Full entry → or outcome? Why does the distinction matter?
Show answer
It is an output (a direct product of the activity). The distinction matters because outputs show the program ran but do not demonstrate change in participants; claiming effect from attendance alone is a classic error.
Why can a pre/post design describe change but not prove the program caused it?
Show answer
Many things change over time without the program: participants mature, other services intervene, seasons or policies shift. Without a comparison group experiencing the same period but not the program, the change cannot be attributed to it.
Give an example of how mixed methods could improve an evaluation that found attendance declining.
Show answer
Example: quantitative data show attendance falling after week three; interviews with dropouts reveal the sessions conflict with work schedules and content felt irrelevant — leading to rescheduling and content changes. Any answer connecting "how much" with "why" is correct.
Study tools & related lessonsKey vocabulary · Related
Key vocabulary
- Formative evaluation
- Evaluation during development to shape the program
- Process evaluation
- Evaluation of whether the program was delivered as planned
- Outcome evaluation
- Measurement of short- and medium-term changes in participants
- Impact evaluation
- Assessment of longer-term effects and their causes
- Logic model
- A diagram of inputs → activities → outputs → outcomes → impact
- Output
- A direct product of activities (sessions held, people served)
- Outcome
- A change in participants (knowledge, behavior, health)
- Indicator
- A specific measurable sign of an outcome
- Validity
- The degree to which a measure captures what it claims
- Reliability
- Consistency of a measure across uses
- Mixed methods
- Combining quantitative and qualitative data
- Continuous quality improvement
- The cycle of plan, implement, evaluate, adjust
Sources & references
This lesson was adapted from the open educational references above; their licenses and attributions are preserved. See Copyright & Licensing.
Educational content only. It is not medical, legal or professional advice. Found an error? Tell us.