Don't let them Pick(er) on you

I have a message for my colleagues and co-workers: don’t let them Pick(er) on us.

(Background: For those who do not work in the medical environment, patient satisfaction has become part of our government’s definition of quality care and therefore a tactical and economic goal for institutions. NRC Picker is a national organization that measures and reports on patient satisfaction. Practices and their employees are graded on patient satisfaction, and compensation is put at risk as a motivator.)

At a recent practice staff meeting, our providers and staff had a very good discussion about some of the things that make it harder for us to do a good job caring for our patients. Lots of important problems were identified and insightful suggestions made. As part of this, we also talked about the fact that our most recent patient satisfaction scores were disappointing and had not improved from the previous year. The more I thought about it, the more convinced I became that any discussions of patient satisfaction as measured by Picker surveys is a distraction from the more important task of improving quality.

Let me start with an example, illustrating just how badly broken the use of Picker scoring is. Imagine you are starting your junior year in high school and the English teacher tells you how she will grade you. Vocabulary will count 20% of your final grade. Every week she will give you a vocabulary quiz of 20 words, asking for a definition and a sentence using the word. There will be a total of 800 words in 40 quizzes. At the end of the year, the class will be ranked according to the number of correct answers, and only the top quarter will pass and get full credit for vocabulary. The other three quarters of the class will fail vocabulary and get a zero, making their best possible English grade an 80. (I hope none of you actually had a teacher this bad.) Think about it:

• Vocabulary is only one part of English. A vocabulary score doesn’t reflect one’s understanding of theater, ability to write or understand poetry, understanding of literary criticism, understanding of grammar or ability to write clearly.
• This system doesn’t even measure the quality of vocabulary.  One might be able to do well on the quizzes but use common words badly. A student might learn 2000 new and useful words and get no credit for it at all, because 1/4 of the class learned 2010 new and useful words. In fact, a huge vocabulary may impair the ability to write clearly, and too much effort spent on vocabulary may detract from the time spent learning more important English skills and concepts.
• Vocabulary is easy to measure, which explains why it is often used this way. This is like the man looking for his missing wedding band on the living room floor because it is easier to look there than in the bathroom where he lost it.
• Percentiles compare students to other students and cannot measure any individual’s effort, competence, or improvement. If you only want the top 10 percent rewarded, percentiles work. If you want people to be rewarded for their ability or skill, percentiles are the wrong tool.
• Scores will cluster at the top end, and there will be no MEANINGFUL vocabulary difference between the student who got 770/800 and the student who got 740/800. (And certainly no English skill difference.)
• By definition, 3/4 of students fail, regardless of how how hard they work, how much English they know, or even how good their vocabulary is and how much it improves.
• If 25 students get 790/800 correct (98.7%), the next student in the ranking, who gets 785 correct (98.1%), fails. In the extreme case, if every student gets every vocabulary word correct, no students are better than any other students, no students are in the top 25th percentile, and all students fail - despite all getting perfect scores.
• From an educational and behavioral/motivational perspective, this is a system designed to minimize success and maximize frustration for individuals within the system.

This is exactly how the Picker system is set up. It uses paper and pencil surveys filled out weeks after a visit to rank practices based on how many patients gave top scores (only a 9 or 10 out of 10 counts), and then fails the bottom three quarters of practices. As clinicians and practices work to improve their performance, scores bunch closer and closer together at the top end, and there is no MEANINGFUL difference between the 90th percentile and the 60th percentile. (Percentile ranking is only useful when scores are evenly distributed, such as in a bell-shaped normal distribution. The more skewed the scores, the less informative percentile ranking becomes.) At least with the vocabulary quizzes, students get their corrected quizzes back and can learn the words you missed. The quizzes provide actionable educational information. With Picker, practices get very delayed feedback without useful of specifics. Individual clinicians and practice staff have no way of knowing which interactions with which patients needed to be improved in what ways. This is like the soccer coach telling his team they need to score more goals but not teaching them how to score more goals.

(As a side note,  the Picker paper and pencil forms are sent out weeks after the visit. How accurate are they? Do you remember what you had for lunch last week? What kind of day you had two weeks ago? I thought not.)

I am committed to doing the absolute best job I can in patient care, and I know my staff pretty well. They are equally committed to high-quality, reliable, responsive, patient-centered care. I owned and ran a practice for more than 30 years, and I know with certainty that the majority (probably 90-95%) of the time dissatisfaction or error occurs not because someone doesn’t care or isn’t doing their best, but because the system is constructed in such a way that doing the best one can at the job (as it is defined with the resources and training one is given and within the constraints imposed on one) doesn’t allow the individual to do any better.  In short: ITSS.  (It’s the stupid system, or it’s the system, stupid.)

The solution is NOT to talk about how we as individuals are failing to provide a good patient experience. The solution is to talk about how the system does not allow us to provide as good patient experience as the patient deserves and we intend.

Patient satisfaction scores from Picker surveys can be one piece of information we use as a tool to track the job we are doing, but Picker scores are not important enough to spend a great deal of time and energy on, and we should not let ourselves be distracted (by focusing on Picker scores) from working on improving actual quality, or bullied into thinking we  - as individuals - own this problem. 

My message to my colleagues and co-workers is this. We are all committed to making our practice a stellar place to work and the best place in Central Maine to get health care. Let’s focus on quality. Let’s feel free to look for and talk about the things that don’t work was well as they should. Let’s agree to talk about these things without being judgmental and without taking offense. Let’s take ownership of trying a wide range of solutions to see what works and what doesn’t. Lets try lots of little things and see how they work. (And remember, if we try too many things at once, ti can be hard to tell which ones worked and which ones made things worse.) Let’s push back hard at the institution to give us the resources we need to do a good job - or at least, to remove the barriers the system tends to create. If this improves our Picker scores, I have two answers: 

1. Great job of improvement!
2. Picker scores better? So what?  

And if we work hard collaboratively to improve, but it is not reflected in our Picker scores, I have two answers: 

1. Great job of improvement!
2. Picker scores worse? So what? 

Stay focused on the patient. Stay focused on quality. Stay focused on making things better. Be hard on ideas but not on people. Remember that you are doing the best you can, and that I appreciate and respect you for that.

And, above all, don’t let anyone Picker on you.  ITSS.