Plus-Delta, Scaled Up

I don’t remember when I learned about the “Plus-Delta” event evaluation tool; I have used it for a long time.

After a meeting or training session, I give people an opportunity to offer statements of two kinds: (1) those aspects of the event that went well and that we should retain for future work (plus); (2) those aspects of the event that we should change to improve the experience in the future (delta).

For groups with fewer than a dozen people, I give everyone a chance to speak in turn. People can pass or upvote a statement already made.

For larger groups, I just open the floor for comments:  no debate of criticisms though as usual I will ask for clarification if I don't understand the comment.

When the group in the event or meeting has not worked together before, plus-delta helps everyone learn how to work together better. The photo of the plus-delta at the top of the post is from a new group; the evaluation helped us act on our prior agreement to collaborate and improve our working relationships.

For a group that will meet again, the session planners have an opportunity to address the delta items.

An IHI project team I’m on that works with a federal agency on a training program is particularly disciplined about incorporating the plus-delta feedback from session to session. In particular, the meeting planner reports back to the group at the start of the subsequent session what we have done to address the deltas.

Extending Plus-Delta: Patient Experience Data

You can ask the plus-delta questions of anyone, to evaluate any experience.

The down-side of the questions: they are open-ended, which makes summary analysis harder than counting pre-defined answers that are typical in experience surveys.

Farrokh Alemi has published several papers* (in 2008, 2009, 2012 and 2014) that build on the plus-delta question to assess patient experience.

His research suggests that answers to the plus-delta question can be mined automatically to assess multiple aspects of care like those covered by the CAHPS system in the United States.

He provides evidence that plus-delta questioning yields higher response rates than traditional formal surveys—at least partly because there are just two questions. In his survey structure, a delta (item needing improvement) is treated as a complaint.

Professor Alemi has proposed a “time to next complaint” control chart to allow organizations to characterize satisfaction and to detect changes over time—the chart is based on the geometric distribution, discussed by Provost and Murray (2011), The Health Care Data Guide, Wiley: New York, Chapter 7 and is actually a “cases until next complaint” chart when used with the plus-delta questions.

Professor Alemi’s work deserves wider study and application, especially as organizations face lower response rates and challenges in parsing response bias in multi-question surveys.

Rethinking satisfaction surveys: minute survey.
Alemi F, Badr N, Kulesz S, Walsh C, Neuhauser D.
Qual Manag Health Care. 2008 Oct-Dec;17(4):280-91.

Rethinking satisfaction surveys: time to next complaint.
Alemi F, Hurd P.
Jt Comm J Qual Patient Saf. 2009 Mar;35(3):156-61.

Feasibility of real-time satisfaction surveys through automated analysis of patients' unstructured comments and sentiments.
Alemi F, Torii M, Clementz L, Aron DC.
Qual Manag Health Care. 2012 Jan-Mar;21(1):9-19.
https://tellmymd.com/images/SetimentAnalysisPatientComments.pdf

An alternative to satisfaction surveys: let the patients talk.
Alemi F, Jasper H.
Qual Manag Health Care. 2014 Jan-Mar;23(1):10-9.
https://tellmymd.com/Images/An-alternative-to-satisfaction-surveys.pdf