Placeholder Image

Subtitles section Play video

  • >> Welcome back to our video series

  • on evaluating health adaptation for a changing climate.

  • In this video, we illustrate evaluation best practices

  • for describing health adaptation work using our real world scenarios.

  • Describing the program is the second step in the CDC evaluation framework.

  • In this step, we describe clearly what our program does and depict visually

  • with a logic model the outcomes we expect to achieve as a result of our activities.

  • In step two, stakeholders use a participatory process to create a logic model,

  • which visualizes how a health adaptation is expected to work.

  • We know that health adaptations are complicated by time, scale, and complexity,

  • creating logic models to describe them as equally complicated.

  • For example, simple logic models may suggest linearity and causality where it does not exist.

  • Similarly, depicting complicated relationships, or influential factors

  • or conditions using logic models can be challenging.

  • Overcoming challenges and illustrating complex health adaptation cause

  • for following useful design practices, such as using diverse design elements,

  • integrating context, and making assumptions explicit.

  • We can convey complicated adaptations in a number of ways, like placing time

  • or scale boundaries in and around our model.

  • Using different types of lines and arrows allows us to show relationships

  • and the strength of those relationships.

  • A bolder or more solid line can convey a stronger relationship,

  • or hypothesize relationship between the items in the boxes.

  • Varying shades of color and different shapes or symbols can convey different categories

  • or intensities of connections or relationships.

  • It is also important to integrate context into the model.

  • The context of an adaptation is critical in enabling

  • or hindering activities, outputs, and outcomes.

  • We can improve our evaluation by incorporating this information by, for instance,

  • including projected climate data, or vulnerability indices

  • in our logic model if they are relevant.

  • Finally, we can be very explicit in noting any assumptions we have

  • between the boxes or nodes in our logic model.

  • This will lead to insights about how our stakeholders expect the adaptation to work.

  • Let's now check in with our practitioners to see how they are using these practices

  • to describe their health adaptations, the second step of the evaluation framework.

  • If you would like to revisit our practitioners' evaluation journeys so far,

  • please refer back to the previous videos in this series.

  • To describe the new city greening project,

  • Cassandra's logic model uses two of our design principles.

  • She is color coding her outcomes and using rainfall projections

  • to illustrate the project's potential impacts.

  • By color coding according to different projection scenarios,

  • Cassandra is able to show stakeholders the different impacts they might expect

  • to see with the greening project.

  • Further, this helps her explain how all outcomes in the logic model, especially health outcomes,

  • may be affected by different rainfall levels.

  • These design principles also help demonstrate the point

  • at which greening may not effectively manage stormwater.

  • For instance, if the city begins experiencing higher rainfall levels,

  • then city officials may need to revisit decisions about the usefulness

  • of greening for stormwater management.

  • Instead, they may decide that resources should be redirected towards more sweeping efforts,

  • such as installing larger pipes to better manage the volume of stormwater.

  • Meanwhile, at her state health department,

  • Elaine continues her evaluation work on wildfire preparedness.

  • She is using shapes and design elements

  • to describe how the multitude program crosses different scales.

  • Because her program includes three major interventions; outreach, training,

  • and resource assistance, Elaine updates her larger model

  • to show how the interventions fit together.

  • She then breaks it into three intervention specific logic models.

  • To better describe each specific intervention, Elaine uses different shapes

  • to create a separate, smaller logic model, depicting how each intervention intends

  • to achieve its specific outcomes.

  • This nested approach provides a clearer picture of how the program is expected to work.

  • This design also allows Elaine to show how data from different interventions can feed

  • into the same outcome category, such as knowledge increase or behavior change.

  • Now let's check in with Jackson at the county health department

  • to see how his training evaluation plan is progressing.

  • After engaging a few additional stakeholders,

  • Jackson and the work group conduct a logic modeling session to map

  • out how they expect their home health aide intervention to unfold.

  • They start by writing down all the activities and intended outcomes on sticky notes

  • and placing them on the whiteboard to show how they think the program will work.

  • Jackson then engages his group in a discussion about assumptions.

  • For example, conducting trainings won't automatically lead to an increase in knowledge.

  • Underlying factors often hold our logic model pieces together.

  • The group works to identify underlying assumptions between two key outcomes;

  • home health aides' increase in knowledge, and the adoption of a new behavior,

  • such as identifying symptoms of heat related illness.

  • As the group begins examining the various parts of the logic model

  • and generating their assumptions, they ask themselves,

  • to what extent are these assumptions valid?

  • Following one of the design principles, Jackson suggests making these assumptions explicit

  • in the logic model by using dashed lines to show the areas of uncertainty.

  • To further capture the discussion, he adds notes to make their assumptions explicit

  • and places them at the bottom of the financial logic model.

  • Clearly noting this information will help the group decide on an appropriate set of standards

  • and indicators later on in the evaluation process.

  • Using the best practices described in this video can allow you to tackle common challenges faced

  • by practitioners in step two of the evaluation framework.

  • CDC provides many useful resources on describing programs.

  • For more information, please visit our climate and health evaluation webpage.

  • In the following video, our three practitioners will begin step three of the evaluation cycle,

  • focusing the evaluation design for their health adaptation work.

>> Welcome back to our video series

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it