Causal Inference: Guidance for Quality Improvers
Let’s consider a situation related to the sealant work discussed in many of my previous blogs. A dentist working at the East Clinic has found that specific changes to policy and workflow now give high rates of sealants applied to children’s permanent molars. It looks like a causal effect.
Here's a causal diagram that represents the causal effect, “A causes Y”:
Remember that in a causal diagram, the absence of arrows represents strong belief. The “A causes Y” diagram states that no other factors contribute causally to the outcome.
Refining the Causal Theory
Could clinic-specific factors affect the way the policy and workflow change is deployed and also directly affect the outcome?
In Figure 2, we admit that there may be more going on than just A and Y.
Furthermore, in graph analysis language, diagram 2 shows a ‘back-door’ path Y to L to A that means A and Y could be associated even if A does not cause Y. We need to be cautious about claiming that policy and workflow change will cause the observed outcome in clinics other than East.
Discussion with an enthusiastic dentist
I spoke with the dentist leading improvements at East Clinic. He saw good results from his change in sealant policy and workflow and wanted to spread to another ten clinics in his organization.
We talked about his causal thinking and concluded together that diagram 1 did not match reality. Diagram 2 seems more reasonable.
If the second diagram describes reality better than the first, then it’s easy to start thinking about clinic-specific factors L like:
Clinician enthusiasm for changes in care
Process discipline to maintain new changes week after week
Skill in doing the sealant work
Equipment availability
Physical environment/flow of patients in the clinic
IT system performance and skills
Patient factors like trust, preferred language, ability to keep appointments.
This list of factors seemed to cover a lot of possibilities and primed us to adjust expectations about a rapid spread and success of East’s new sealant policy and workflow.
Thinking with a causal diagram
Experienced managers and improvement consultants already know that spread of an effective management change from one clinic to another usually requires testing and adaptation to account for variation in clinic environments.
What’s new here? The simple causal diagrams helped me to quickly focus our conversation on factors that may matter; the diagrams set the stage to discuss what might be unique about East clinic, thus opening our thinking to consider a strategy for spread.
The second causal diagram captures the concept of confounding without statistical jargon. We just talked about factors that might differ across the eleven clinics and could drive the improvement observed at East.
Finally, the second causal diagram reminded us to begin modestly with a claim that the change tested at East clinic will cause improvement across all ten clinics; we need to test the change and observe performance, clinic by clinic before we can make a strong causal claim.
