Michael Quinn Patton, a leading influencer and thinker in the realm of evaluation, was recently in Edmonton to give a presentation on Developmental Evaluation.  The day was packed to the rafters with learning but here a few things I found most salient.

The Evaluation Gap

The way programs are experienced is often disconnected from how they are evaluated.  Logic models are designed to emphasize the way the program is experienced by the individual and neglects how the program is experienced by the group.  In particular, relationships and social capital generated as a group is not usually a measurable outcome articulated in the logic model.  How well the program works for the group can be overlooked by how well the program worked for the individual in the group.

Lessons vs. Lessons Learned

Lessons are knowledge.  Lessons Learned are actions based on the new knowledge.

Formative vs. Summative vs. Developmental

Evaluation in a reasonably predictable context using previously tested models occurs formatively and summatively.  Formative assesses project/program progress as it unfolds.  Summative assesses the overall performance of the project/program to determine if it will be continued or altered.  It answers the question; How do we improve the model?

Evaluation in a dynamic, unpredictable context without previously tested models requires a more flexible, iterative approach.  Developmental evaluation is a constant process of trying various approaches and learning lessons via case-based reflective practice.  It answers the question; How do we change what is occurring?

A couple other Patton nuggets from the day:

“Emergence is when people find each other and opportunities emerge from the connection”

“You can have specific outcomes and targets only when you know how to produce them”