Prediction is the Problem, Ron Moen’s presentation at the 2012 Annual Deming Conference.
“Planning requires prediction. Prediction requires a theory.”
In the video Ron Moen talks about the Associates for Process Improvement model for improvement. The enhanced PDSA cycle includes an explicit focus on prediction in the planing phase and then evaluation of that prediction in the study phase. As I have written previously The Improvement Guide (by the API consultants, including Ron Moen) is the best book to help improve the application of the PDSA cycle, in my opinion.
The book he discusses in the presentation is Quality Improvement Through Planned Experimentation (by 3 of the same authors as the Improvement Guide).
Ron touched on the idea of the fallacy of using treating data from a process as if it were sampling data on an entire population (sampling a river [ongoing process] and treating your data as if you were sampling a pond [fixed population to sample]). This is a fallacy that still traps and confuses many people today. Ron referenced Mike Tveite’s talk on this topic, which I posted about previously.
One nice detail from the talk was when Ron noted the footer of the paper Dr. Deming handed out in 1989 (when Dr. Deming was 88 years old): “comments and help appreciated.” Dr. Deming worked and continued learning for his entire life. He was constantly improving his theories on management based on what he learned from applications in organization and from the insights of others.
As he mentions late in the clip the focus of experimentation is to increase subject matter knowledge and statistics is used to enhance that learning. Often people can be distracted by the statistics and may end up focusing on the statistics themselves. Keeping the focus on what we are learning about our process is important and one of the big differences I see when talking to experts (like Ron Moen, George Box, Don Wheeler, Lynda Finn…) and most others (who are too focused on statistical tools instead of just using the tools to gain knowledge).
It seems to me this is often because experts have mastered the tools and don’t have to think about the details of using the tools. They are focused on the big picture and grab whatever tool is needed and quickly apply it to the matter at hand. Many others get so focused on the details of the next step they lose track of the big picture.
I like this comment on a post from my Curious Cat Management Improvement blog: Management is Prediction (by another API consultant):
The prerequisite for a prediction is some theory in use. In using the PDSA cycle, is important to frame questions that help people explicitly state their theory, especially if a team in involved. Merely answering yes or no, or guessing, does not make the theory explicit. I have seen instances where people will predict “yes” or “no” for two different reasons. The act of making the prediction can change how we collect data or run a test. Richard Feynman commented, “Science begins and ends in questions.” So should the science of improvement. Questions lead to predictions (theory being explicit), this leads to a plan for data collection and a test. Built into the PDSA cycle is the logic of deductive and inductive learning; Plan to Do (deductive) Study to Act (inductive). This iterative learning process was explored in Statistics for Experimenters by Box, Hunter and Hunter.
Related: On Probability As a Basis For Action by W. Edwards Deming – Circling Back: Clearing up myths about the Deming cycle and Seeing How it Keeps Evolving by Ron Moen and Cliff Norman – The Development of Deming’s Management System – Design of Experiments: The Process of Discovery is Iterative (webcast with George Box) – Write Down Predictions to Improve Learning (Ackoff) – Theory of Knowledge