Michael Quinn Patton Visits Edmonton

Michael Quinn Patton, a leading influencer and thinker in the realm of evaluation, was recently in Edmonton to give a presentation on Developmental Evaluation.  The day was packed to the rafters with learning but here a few things I found most salient.

The Evaluation Gap

The way programs are experienced is often disconnected from how they are evaluated.  Logic models are designed to emphasize the way the program is experienced by the individual and neglects how the program is experienced by the group.  In particular, relationships and social capital generated as a group is not usually a measurable outcome articulated in the logic model.  How well the program works for the group can be overlooked by how well the program worked for the individual in the group.

Lessons vs. Lessons Learned

Lessons are knowledge.  Lessons Learned are actions based on the new knowledge.

Formative vs. Summative vs. Developmental

Evaluation in a reasonably predictable context using previously tested models occurs formatively and summatively.  Formative assesses project/program progress as it unfolds.  Summative assesses the overall performance of the project/program to determine if it will be continued or altered.  It answers the question; How do we improve the model?

Evaluation in a dynamic, unpredictable context without previously tested models requires a more flexible, iterative approach.  Developmental evaluation is a constant process of trying various approaches and learning lessons via case-based reflective practice.  It answers the question; How do we change what is occurring?


A couple other Patton nuggets from the day:

“Emergence is when people find each other and opportunities emerge from the connection”

“You can have specific outcomes and targets only when you know how to produce them”

Podcast Pick: On Average by 99% Invisible

I’m a huge podcast fan and sometimes I come across an episode that I feel compelled to share.

On Average by 99% Invisible (20 min) is an informative bit of  research illustrating how the mathematical concept average can solve problems……or create them.

It starts by outlining the historical origins of how the average measure was discovered and subsequently used.  For example, it helped solve a clothing issue for President Lincoln by creating the Small, Medium and Large sizes we see in stores today.  Later  it illustrates how designing for the average had disastrous outcomes for the United States Air Force.

Can you think of any examples in your life/work where designing for the average has been a help? or a hindrance?