Shows
Adventures In Evaluation PodcastThe Big Lebowski and Evaluation
What does evaluation have to do with the movie The Big Lebowski?
Well, the answer is "a lot" if you're up on this cult classic and number one favourite movie of James, aka "The Dude".
Before you listen to this podcast we strongly suggest you view it first so you don't miss any of our keen insights.
Cheers,
Kylie & James
P.S. Here is the photo of Kylie recording in the Starbucks bathroom while on family vacation.
2014-06-2729 minAdventures In Evaluation PodcastThe Future of Evaluation: A Chat With Michael Scriven
This week we're thrilled to have Michael Scriven, frequently called "the father of evaluation", as our special guest. Michael chats about his chapter on "The Foundation and Future of Evaluation" in the new book, The Future of Evaluation in Society: A Tribute to Michael Scriven. Check it out here or subscribe to our podcast on iTunes for free.
For more details on Michael's work you can visit his website here.
Feel free to leave us your comments below, and don't forget to rate us on iTunes to give us some feedback!
Enjoy,
James & Kylie2014-05-0436 minAdventures In Evaluation PodcastWestern vs. Development Evaluation: Commonalties and DifferencesIn this episode we explore some of the differences between typical Western-style evaluation and the monitoring & evaluation (M&E) that occurs in an international development context. To help us, we invited two guests with experience working in both areas, Ian C. Davies and Tom Pellens.
Here are some of the links we mention:John Mayne and Contribution Analysis
Jennifer Greene and Mixed Methods
European Evaluation Society 2014 conference
Below is the cartoon mentioned by Terry Smutylo and Daniel Morales-Gomez.
Until next time,
Kylie & James
2014-04-1238 minAdventures In Evaluation PodcastFun in Cameroon at AfrEA 2014 and The Quest for Exciting Journal ArticlesIn this episode we present a mixed bag where Kylie debriefs her most excellent week at the African Evaluation Association 2014 conference and James takes us on his search for a "non-boring" evaluation journal article.
Fun on the Conference Shuttle Bus
Shout outs to Serge-Eric Yakeu Djiam, Jennifer Bisgard, and all the hard-working volunteers at AfrEA2014 and the Cameroon Development Evaluation Association for a great conference.
Below are some of the articles that James found in his search: American Evaluation Association Task Force Statement on ‘What is Evaluation?’ (2014)
Fournier, D. M. (1995). Establishing evaluative conclusions: A distinction between gene...2014-03-2320 minAdventures In Evaluation PodcastNew Year's Resolutions for Evaluators!
Happy New Year Everyone!...okay...we're a little late with that...but we have a short new podcast!
Many of us have personal New Year's resolutions, but what about professional ones? In this episode James and Kylie share their evaluation resolutions for 2014.
Got some of your own? We'd love to hear them so leave us a comment below.
Cheers
Kylie & James
P.S. Kylie just thought of another resolution she forgot to mention - use a systems lens!2014-02-0116 minAdventures In Evaluation PodcastMobile Data Collection in EvaluationIn this podcast we chat with Kerry Bruce, Director of Results and Measurement at PACT, and an all-round expert on the use of mobile technology for data collection. Here are a couple of resources:Kylie is curating a Pinterest Board of resources in Mobile Data Collection here. If you have any additional suggestions please let us know.
Below is the detailed cost comparison we discuss from Tanzania courtesy of Daisy Nathan Kisyombe and PACT.
Enjoy,
Kylie & James
2014-01-0936 minAdventures In Evaluation PodcastEvaluation Rubrics with E. Jane DavidsonHave you joined the rubrics revolution? In this podcast we chat with Jane Davidson of Real Evaluation about the use of rubrics in evaluation. If this piques your interest, we have a slew of resources for you:slides from the AEA 2013 conference on rubrics:
Jane's Evaluation Rubrics: What, Why, and How
Kate McKegg's The Value of Rubrics in Messy Non-Profit Evaluation Contexts
Better Evaluation and Judy Oakden have a new document on rubrics, Evaluation Rubrics: How to Ensure Clear and Transparent Assessment That Respects Diverse Lines of Evidence
You can find a number of real-life rubrics from the New Zealand Ministry...2013-12-0834 min
Adventures In Evaluation PodcastReflections on AEA2013 in Washington, DC
Couldn't make it to AEA2013? In this episode we share our own personal highlights of the conference. By the way:the Twitter hashtag for Michael Quinn Patton is #omgmqp.
Sheila B. Robinson gives a great summary on her blog here.
the evaluation typology map that Kylie mentions is here.
Finally, don't forget to check out Chris Lysy's hilarious cartooning of the conference.
Enjoy,
James & Kylie
2013-11-0725 minAdventures In Evaluation PodcastSheila B. Robinson and AEA365This week we talk to Sheila B. Robinson about the great AEA365 Blog!
cheers,
Kylie and James
2013-10-3019 minAdventures In Evaluation PodcastPresenting Data Effectively: An Interview with Stephanie Evergreen
Stephanie Evergreen's new book on data visualization came out yesterday, so we asked her to join us again on the podcast to give us some highlights plus her thoughts about the role and future of data visualization in evaluation.
You can purchase a copy of the book here.
See you at AEA!
Kylie & James
2013-10-0721 minAdventures In Evaluation PodcastAnyone for a Horse Race?
Anyone in the mood for a horse race? Today we tackle the issue of different and competing evaluation approaches, models, types, whatever you want to call them, plus the challenges of inconsistent terminology in our field.
Here are the two articles we mention by Paul Duignan:
Terminology in evaluation: Approaches, types (purposes), methods, analysis techniques and designs
Untangling Evaluation Terms – Discussing Evaluation 'Types' With Clients Often More Useful Than Evaluation 'Approaches'
Here's the Evaluation Typology map that Kylie mentioned.
If you're interested in the free Evaluation Glossary Mobile App you can download it here.
As usual, we'd love to...2013-09-0225 minAdventures In Evaluation PodcastGame Changers for Evaluation: A Chat with Hallie Preskill
This week we're thrilled to have Hallie Preskill join us to give a shortened version of her keynote speech at CES2013 in Toronto.
Here are links to some of her current work, plus the Developmental Evaluation graphic she mentioned:Evaluating Social Innovation
Breakthroughs in Shared Measurement
Buiding Evaluation Capacity
Reframing Evaluation Through Appreciative Inquiry
Developmental Evaluation graphic (full article)
And here is this link to the book she mentioned on Big Data.
Don't forget to leave us your comments below.
James & Kylie
2013-08-0142 minAdventures In Evaluation PodcastThe Changing Role of Evaluators: Reflections from the CES2103 ConferenceAre you a technical specialist, or a social change agent? Or as Dan Wilson at the Ontario Trillium Foundation says, "Are you part of the Rebel Alliance, or a Jedi Knight?"
In this episode, we discuss the changing role of evaluators based on provocative keynotes from Hallie Preskill and John Gargani the CES2013 conference in Toronto.
As promised, here are some of the links we discussed:Jane Davidson's ebook, Actionable Evaluation Basics: Getting succinct answers to the most important questions
Stephanie Evergreen's webinars and book
SROI Canada
John Gargani's Evaluator Doll John Gargani's slide on evaluators as social change agents.
2013-07-0523 minAdventures In Evaluation PodcastThe Launch of the Evaluation Mentoring Canada InitiativeIn this episode Kylie interviews James and Jane Whynot about their super cool initiative, Evaluation Mentoring Canada. Check out the website, and sign up: http://evaluationmentoringcanada.ca/.
Cheers,
Kylie & James
Kylie and her first mentor, Dr. Anita Myers, at CES2013.
2013-06-1526 minAdventures In Evaluation Podcast2013 CES Conference PreviewHi Everyone, This week we've got Dawn Campbell-Borland and Dayna Albert as guests giving us a sneak preview into this year's CES conference in Toronto, June 9 - 12th. This year's theme is "Evaluation Across Boundaries". They have some great workshops, keynotes, and presentations lined up, so have a listen!
For more information about the conference please visit http://cestoronto2013.ca/.
Cheers, Kylie & James
2013-05-1232 minAdventures In Evaluation PodcastThe CES Evaluator Competencies: A Chat With Heather BuchananIn this episode we chat with Heather Buchanan, evaluation volunteer extraordinaire about the history, development, and use of the Canadian Evaluation Society's Competencies for Evaluation Practice.
Here are some related links:
Competencies for Evaluation Practice (detailed version)
CES Professional Designations Program
Establishing Essential Competencies for Program Evaluators
Cheers,
James & Kylie2013-04-2722 minAdventures In Evaluation PodcastMaking the Leap: Evaluation Consulting with Gail BarringtonThis week we chat with Gail Barrington, Evaluation Consultant and author of Consulting Start up & Management: A Guide for Evaluators & Applied Researchers.
Plus, here's a great little AEA365 post about small-sized consulting.
Cheers,
James & Kylie2013-04-0631 minAdventures In Evaluation PodcastOh My God It's March: Dealing with March MadnessIt's March again, and evaluators everywhere are frantically racing to meet multiple deadlines. (In fact, Kylie almost forgot to post this episode herself.) Have a listen to James and Kylie discussing some of their techniques for dealing with March madness.
How do you deal with it? Leave us your comment below.
2013-03-1629 minAdventures In Evaluation PodcastHey, Where Did My Stats Go?Some of us evaluators find ourselves using our stats straight away out of school, and for various reasons some of us don't. In this episode we discuss the different levels of use among evaluators and what is really acceptable in terms of statistical knowledge.
Here's the link to Paul D. Ellis' great little e-book, Effect Size Matters.
AEA often has eStudy courses on statistics, and here are some other links for online refreshers:
Rice University's Online Statistics Education: An Interactive Multimedia Course of Study
Johns Hopkins' Introduction to Biostatistics 1
Coursera/Princeton University's Statistics 1
UDACITY's Introduction to Statistics
This list is by...2013-03-0231 minAdventures In Evaluation PodcastHumour and EvaluationThis week we're pleased to have Chris Lysy, evaluation cartoonist, as our special guest. Chris talks with us about the use of humour in evaluation. To see more of Chris' evaluation funnies visit his website here.
And here's a link to another example of humour in evaluation: Roger Miranda's Eva the Evaluator.
2013-02-1728 minAdventures In Evaluation Podcast'Soft' Skills for Evaluators: Facilitation and MoreThis week we talk about facilitation and it's role as an essential skill for evaluators. Here are some of the resources we mention:
book by Sam Kaner
International Association of Facilitators
Government of Canada's Centre for Intercultural Learning
If you have buckets of money and want a skilled instructor, check out this offering at Hollyhock here. This one is definitely on my To Do list one year.
plus....
we'll be discussing the ARECCI ethics tool in a future episode, but here is the link for now
a photo of Kylie's charging station for her 12 offline mobile data collection tablets.
All...2013-01-3136 minAdventures In Evaluation Podcast10 Things You Didn't Learn in Evaluator SchoolToday we chat with evaluator Ann K. Emery about her recent AEA365 blog post for on "10 Things You Didn't Learn in School".
Apologies up front for the poor sound quality in this particular episode while Kylie Skypes-in from Africa, and for her rambling after a hard day at work and several glasses of wine later.
The management blog that Ann mentions is Ask a Manager.
And here is John Gargani's blog post on jargon.
Enjoy,
Kylie & James2013-01-1026 minAdventures In Evaluation PodcastEvaluation Proverbs, Prepping for Results, and Mobile Data CollectionHi folks,
This week we again chat about a number of miscellaneous items including helpful proverbs for explaining evaluation in different cultures and mobile data collections.
Here are the links to the West African Proverbs for evaluation and the 30 Ideas for Building a Culture of Evaluation. You can join the AEA365 Tip A Day here and all the Evaluator Christmas Carols here. Apologies for Kylie's singing.
Cheers,
James & Kylie
ps-bigger apologies about my countertop ramblings!!~James :)2012-12-2129 minAdventures In Evaluation PodcastEverything but the Kitchen SinkIn this episode Kylie and James debrief their respective weeks in evaluation and end up covering a multitude of subjects including re-discovering one's stats skills, evaluator objectivity, and reporting to senior execs.
To see the Chris Lysy cartoon we reference click here.
Here's an example of the Post-it note technique Kylie refers to:
And the stock photo:
"Help, my son is a stock photo photographer..."
And here's the link to the Government of Canada's Treasury Board report, "2011 Health of the Evaluation Function" that we...uh...didn't get around to discussing.
Until next time,
James and Kylie2012-12-1230 minAdventures In Evaluation PodcastBudgeting for EvaluationHi Everyone,
In this episode we discuss budgeting for evaluation along with the following resources:
WMU's Checklist for Developing & Evaluating Evaluation Budgets-by Jerry Horn
The Pell Institute's Evaluation Toolkit
Kellogg Foundation's Evaluation Handbook
The Program Evaluation Standards F4 & P7
Michael Scriven's EVALTALK Comments
Know any others? Leave us a comment below.
cheers,
James & Kylie2012-11-2226 minAdventures In Evaluation PodcastAEA 2012 Conference Interviews & DebriefNeither James nor I were able to make this year's AEA conference :( ... so in this episode we chat with our most excellent reporters on the ground, Pat Mueller and Amy Germuth. You'll hear their personal highlights of the conference as well as the views of various attendees on what was hot and major take-aways.
What was YOUR major take-away from the conference? Leave us a comment below.
Cheers,
Kylie & James2012-11-0527 minAdventures In Evaluation PodcastTop Reasons Why We Like Being EvaluatorsThis week, James and Kylie discuss why they love being evaluators, and have a big happy group hug.
What are your reasons? Leave us a Comment below.
cheers,
James & Kylie2012-11-0323 minAdventures In Evaluation PodcastCredentialing Evaluators: An American Perspective with Jean KingThis episode we speak with Jean King, Professor and Director of Graduate Studies in the Department of Organizational Leadership, Policy, and Development at the University of Minnesota, on the viability of credentialing in the United States.
Here again is the link to the Canadian Evaluation Society's Professional Designation program.
Let us know your comments about what you've heard by clicking on the Comment link below.
Cheers,
Kylie & James2012-10-1222 minAdventures In Evaluation PodcastSeven Deadly Sins of EvaluationThe name of this episode says it all. Have a listen and let us know what you think.
By the way, here's the scene we mentioned from Barton Fink.
Are we missing any sins? You can add them in the Comments box below.
Cheers,
Kylie & James2012-09-2920 minAdventures In Evaluation PodcastAEA2012 Conference Preview with Susan KistlerOur special guest for this episode is Susan Kistler, Executive Director for the American Evaluation Association, who gives us a sneak preview into this year's upcoming conference (Oct. 22 - 27, 2012) as well as some tips for both newbies and the seasoned attendee.
AEA2012 Conference Website
Enjoy,
James & Kylie2012-09-1431 minAdventures In Evaluation PodcastDevelopmental Evaluation: Perspectives from an Internal vs. External EvaluatorThis week we're talking about doing Developmental Evaluation from the perspectives of both an internal (James) and external (Kylie) evaluator.
We're also interesting in hearing what you think. Add your comments below.
Here are some of the links James mentions:
DE Diagnostic Checklist
Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use
McConnell Foundation DE Resources
Don't forget to leave us your comments below (click on Comments just by the five stars).
Cheers, James & Kylie2012-09-0925 minAdventures In Evaluation PodcastDesert Island Evaluation ResourcesIn this episode we discuss our "desert island evaluation resources". These are resources we would want to have with us if we were stuck on a desert island and....uh...still doing evaluation.
Here are links to some of the resources we mention:
Books by Michael Quinn Patton
EVALTALK Listserv
Michael Scriven's Evaluation of Training Checklist
Periodic Table of Data Visualization
University of Wisconsin Extension Evaluation Resources
Centre for Public Health Systems Science Sustainability Framework and Assessment Tool
AEA365 Tip A Day
Twitter @EvaluationMaven
AEA Public eLibrary
Have any others? Please leave your comment below.
cheers, Kylie & James2012-09-0526 minAdventures In Evaluation PodcastAEA's Potent Presentations InitiativeIn this podcast we feature our first guest, Stephanie Evergreen, Data Visualization Diva and Director of the AEA's new Potent Presentations Initiative (P2i).
As always, note that the views of our speakers do not necessarily represent their employers or clients.
Here are some of the links that Stephanie mentions:
P2i Slide Design Guidelines & Preparation Checklist
Ignite Training Webinar Archive
Stephanie's Blog
Gavin McMahon’s Webinar on Presenter Types
Any comments you'd like to share? Leave them below.
Kylie and James2012-08-1826 minAdventures In Evaluation PodcastEvaluation ReportingHi Everyone!
Here's our 2nd podcast in which we talk about the need for more effective evaluation reporting and some of ways to achieve this.
Below are links to some of the resources we refer to during the podcast:
Better Evaluation Reporting Pinterest Board
Jane Davidson article
Upcoming Evaluation Reporting Webinar
Hinterland Who's Who
We'd love to hear your feedback. Please feel free to add comments below or send us an email.
Cheers,
James & Kylie2012-08-0930 min
Adventures In Evaluation PodcastCredentialing Evaluators: A Worthwhile Exercise?Welcome to our first full length podcast!
In this podcast we discuss the Canadian Evaluation Society's Professional Designations Program and our experience with the process.
For more information on the credentialing program check out the CES site: http://www.evaluationcanada.ca/site.cgi?en:5:6
Enjoy!...and feel free to give us some feedback by adding a comment below or sending us an email to adventuresinevaluationpodcast@gmail.com .
cheers,
James & Kylie (both C.E.!)2012-08-0229 min