These days, I’d never consider trying a new restaurant or hotel without reading the on-line ratings on TripAdvisor or Yelp. I seldom even bother with professional restaurant or travel critics.
Until recently, there was little patient-generated information about doctors, practices or hospitals to help inform patient decisions. But that is rapidly changing, and the results may be every bit as transformative as they have been in traditionally consumer-centric industries like hospitality. Medicine has never thought much of the wisdom of crowds, but the times, as the song goes, they are a-changin’.
Even if one embraces the value of listening to the patient, several questions arise. Should we care about the patient’s voice because of its inherent value, or because it can tell us something important about other dimensions of quality? How best should patient judgments be collected and disseminated – through formal surveys or that electronic scrum known as the Internet? And what are some of the unanticipated or negative consequences of measuring patient satisfaction and experience? All of these questions are being debated actively, and some newly published data adds to the mix.
Traditional Surveys
For the past few years, Medicare has been administering the HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) survey to a random sample of 300-1000 patients discharged from every U.S. hospital. Results are now posted on Medicare’s Hospital Compare website. Starting in late 2012, hospital payments will be on the line, as part of Medicare’s pay-for-performance program, known as “Value-based Purchasing” (VBP).
When I lecture about VBP, I often ask audiences what weights they believe should be given to clinical quality data (process and outcome measures such as appropriate antibiotics for pneumonia or readmission rates) vs. HCAHPS survey results. Physicians invariably give answers like 80-20 or 90-10. I’ve even heard some say 100-0; namely, the patient’s voice should carry no weight. Such responses are usually accompanied by grumbling about how unfair it is to be dinged because of a hospital’s disastrous parking or inedible food.
Medicare has chosen to use a 70-30 ratio. In other words, fully 30% of a hospital’s bonus – or cut – under VBP will be determined by patient survey responses. For a large hospital like mine, our score on a single item (“rate the quality of nurse communication”) could be worth over $60,000 a year.
I’ve written before about how “patient-centeredness” has become nearly meaningless since it means so many different things to different people. But the knowledge that patient experience scores now carry real weight has provided tangible focus to efforts to promote patient-centeredness. For example, UCSF Medical Center now pays employees bonuses based on patient satisfaction scores – and these scores have improved markedly since this practice began. On my own medical service, the patient satisfaction committee now scours our results and has launched a program to observe our physicians as they interact with patients, then provide feedback. There is even a communication checklist that offers something of a script, with items such as Knock/Ask (“Hi, is it ok if I come in”), Concerns (“I’d like to review a few things with you, but first, is there anything you’d like to be sure we talk about today?” ….. “I see. So you’re concerned the headache may be due to a tumor?”) and Check Understanding (“To be sure I’ve been clear, can you just repeat back to me your understanding of the plan?”).
While the idea of scripting can seem inauthentic – such as when the bank teller asks you if you are having a great day or have plans for the weekend – it can be extremely useful. I now use a script of my own when introducing myself to a hospitalized patient. Since many patients and families still don’t quite understand what a hospitalist is or does, I often say something like, “You may get a survey after you leave, asking ‘did you have any sense that someone was in charge of your care in the hospital.’ I hope you’ll answer yes, because that is precisely my job… to be your orchestra conductor while you’re here.” Patients seem to get it.
While today’s HCAHPS survey focuses on the hospital, another survey – currently being pilot tested in two states –will roll out soon, asking about individual doctors. Medicare plans to publish the results of these MD surveys in the next few years. Don’t be surprised if a physician-level VBP plan, incorporating these data, follows in short order.
Web-Based Ratings
While many traditionalists object to the very notion of using patient experience ratings as part of transparency and payment initiatives, these objections were muted when the data were gathered via a well-validated survey, professionally constructed and administered. But that orderly world is being rapidly supplanted by one that centers on web-based ratings, in all their über-democratic, Yelpy glory. Predictably, the squawking is getting louder.
Enough Internet physician ratings sites have popped up to fill a large bubble, perhaps of the dot-com variety. For example, RateMDs, started by the same guy who started the popular RateMyProfessors.com site (where profs are rated by their quality, clarity, helpfulness, and “hotness”), now hosts reviews on more than 1 million docs in the US and Canada. Other sites in this “space” include Vimo, RevolutionHealth, Vitals.com, HealthGrades, and Angie’s List.
Attempting to bring order to this world, in 2008 the UK’s National Health Service launched its own patient ratings portal. Called “NHS Choices,” it allows patients to rate practices and hospitals, but not individual doctors. Comments are screened (“inflammatory” comments are blocked) and practices are encouraged to post responses. A 2010 JAMA article by Lagu and Lindenauer praised NHS Choices and encouraged Medicare to begin experimenting with a similar site.
It would be an understatement to say that the physician community has not been enthusiastic about on-line reviews and ratings. One concern relates to the possibility that the most disgruntled patients would be the one likeliest to complete surveys or enter comments. This concern is exacerbated by the relatively small number of responses per physician on many of the websites.
While these concerns are understandable, emerging data suggests that most reviews, of both practices and doctors, are positive. For example, a recent study of 386,000 physician ratings on RateMDs found that nearly 50% were a perfect 5 out of 5, and only 12 % were below 2 out of 5. Similarly, two-thirds of patients posting on NHS Choices said that they would recommend the practice or hospital to a friend.
A second objection is that ratings would be frivolous, capturing the “hotel” aspects of hospital care but not the substance. In fact, a recent New York Times article, written by an oncology nurse, argued that “we hurt people because it’s the only way we know to make them better… which is why the growing focus on measuring ‘patient satisfaction’ as a way to judge the quality of a hospital’s care is worrisomely off the mark.” I found this argument specious. Yes, there are times we do have to hurt people to help them (invasive procedures or surgery, for example), but that’s true for all hospitals and physicians. Some are undoubtedly better than others at helping patients prepare for the discomfort, minimizing it, and empathizing with and supporting the patient who experiences it. I’d like to know who they are.
In any case, the argument that patients focus on thread counts and arugula is increasingly being poked full of holes. In the recent study of RateMDs, physicians who were board certified, went to highly rated medical schools, and had never been sued for malpractice received better ratings. While disentangling cause and effect is challenging, these results support the notion that patient ratings are capturing other important elements of care.
An even more persuasive study was recently published by a group of researchers at Imperial College London led by Dr. Felix Greaves (I had the privilege of working with this group during my recent sabbatical, and am a co-author). We examined more than 10,000 patient ratings of hospitals (the average hospital received 62 ratings) on NHS Choices. We found that positive ratings correlated with lower overall mortality and readmission rates. Moreover, hospitals rated by patients as cleaner had a 42% lower MRSA rates than those with poorer ratings. Clearly, patients are clued into some central truths about clinical aspects of their care.
Another objection is that ratings might be submitted by individuals – who may not even be patients – with axes to grind. After finding one horrid rating of himself, and few other ratings, on DrScore.com, Dr. Kent Sepkowitz, a Memorial Sloan Kettering ID specialist, gleefully confessed to entering his own ratings on the site. Writing in Slate, he says that after reading the nasty review
… I did what any normal American male under e-assault would do. I stuffed the ballot box. I pretended to be a patient of mine… and talked up my friendly attitude and thoroughness, gushed over the oodles of time I spent examining me, and declared my overall treatment a success. Not to limit the kudos, I also gave high marks to parking availability by my office. [A quick editorial aside: We’re talking about parking on Manhattan’s Upper East Side, so now we’re getting into some really serious fiction.] …. With my unceasing selfishness campaign, I was able to hike my scores to levels that would make my mother and even my mother-in-law proud.
Concerns about fraudulent entries can cut in both directions. I’m reminded of the mini-scandal that hit Amazon.com in 2004, when the Times reported that a glitch in Amazon’s Canadian site briefly revealed the true identity behind some anonymous book reviews. Turns out some, like Sepkowitz, had praised their own work. Others – including several prominent authors – had trashed the books of competitors.
While these concerns are real, they are similarly real for reviews of hotels and restaurants. My sense is that – with large enough numbers – the truth generally wins out. And there are ways to mitigate this potential hazard. Amazon, for its part, now allows readers to vote on reviews (“Was this review helpful to you?”) and to “report abuse.” The solution to problems with voting, it seems, is more voting.
Personally, my greatest concern relates to the potential tension between patient ratings and appropriate care. There will be times when giving a patient with a viral URI an unnecessary antibiotic is the surest path to a happy patient and a good review. One hopes that future quality measures will include not only patient experiences but also other measures of appropriateness and evidence-based care designed to counteract this perverse incentive.
The Bottom Line On Patient Ratings
Several years ago, I needed to see a dermatologist for a skin lesion. I was referred to a doctor in a downtown San Francisco medical office building. I decided to not play the “I’m a doctor” card, but rather to simply take in the experience. After entering his shabby office, I was ignored by the receptionist for about 10 minutes before she brusquely shoved a clipboard in my direction and told me to fill out a form. I was ushered in to see the doctor about 30 minutes after my scheduled appointment. The doctor, an elderly man in a white coat, was clearly in a rush. He barely looked at me while taking my history with staccato, closed-ended questions, leaving no room for nuance or embellishment. He then spent about 45 seconds looking at the lesion in question, looking up to offer a monotonic (and indecipherable, to a lay person) diagnosis and some vague recommendations. He scribbled a prescription, offering no explanation as to its purpose or its risks. Before I could say a word, and after a visit that couldn’t have lasted more than 5 minutes, he turned for the door and was gone. I was pissed.
At the time, there were no surveys to complete and no websites on which to rate his care. I would have drawn great satisfaction from writing a damning review, and suspect that a few of them might have led to a change in his behavior. At least, I hope so.
As we work our way through this new world of patient surveys and ratings, there will be some hazards to overcome and some unfair results to contend with. We’ll need to do all we can to anticipate these problems and mitigate them, and to try to bring some order to a chaotic marketplace. These seem like surmountable issues, and I am confident that the outcome of capturing the patient’s voice and giving it some real weight is sure to be better care.
Bob,
Guidance to the hospital admin is simple. Hold the bill until after the survey is completed, provide enough narcotics to cover the anticipated time for the patient to receive and send in the survey, and rid the hospital of time consuming cognitively burdensome CPOE and EHR machines in order to allow the nurses more bedside time. Finally, the hospitals should hire a greeter to provide daily butler services.
Best regards,
Menoalittle
This seems to me another step in the shift away from medical paternalism (maternalism) which is, I believe, a good thing in itself. I do share the concerns about unfair reviews, and the tendency of the [perhaps unreasonably] disgruntled to be disproportionately represented in these venues, but I work in the world of Hospice and Palliative Medicine and have been living with the Family Evaluation of Hospice Care scores for several years now. When we finally got over the excuse-making and started looking into the reasons for some of the low scores, we found things that needed to be fixed.
I agree our first accountability should be for our outcomes, but even determining those outcomes is dependent to some degree upon the wants and needs of the patient.
I’m glad you brought up the appropriateness argument because I also think that is going to be huge negative consequence. What about when your patient with classic migraine is convinced they have a brain tumor and want a CT which is negative and then demand an MRI. Or the low back pain patient who swears imaging will show something. I’d love to see the study comparing physicians knowing they were being rated vs. physicians knowing they were NOT being rated and see how practice changes.
It is my sense that, as Dr. Wachter points out, if the ‘n’ is large enough the noise will be muted. In addition, the general public is now familiar enough with online reviews that the disgruntled complainers are fairly easily recognized and dismissed. So in a sense, having all those
reviews on amazon.com and yelp is helping doctors in their concerns about being reviewed. What does worry me is an overreaction on the providers’ part, where every individual interaction is viewed as fodder for an adverse review and treated accordingly. Then we would all be complaining about another form of ‘defensive medicine.’
Bravo. Ok, I’ll admit it. Bob, I wished I had written this post. It is strong, balanced, value driven, and yes, evidenced based.
Recently I wrote on this topic. albeit more superficially. an the mixed messages on patient experience ratings and the role of the patient and family in informing our journey.
http://blog.hcfama.org/2012/03/06/patient-safety-week-2012-informing-the-journey-not-changing-the-destination/ I closed the post “Ending where we began, our destination is clear—high-quality, safe and continuously improving health and healthcare centered on, and in partnership with, the patient, the family, the public, and their community. Let us inform and adjust our journey, consider all research in a balanced fashion, and yes, be prepared to scratch our heads along the way.” Your post makes this point brilliantly.
Jim
PS: now to forward your post to EVERYONE!
I appreciate hearing from a provider in support of physician reviews – as cofounder of Kentucky’s business magazine for doctors I typically hear the opposite. Disclosure: As a patient, I tend to agree with my conservative cohorts.
I don’t believe that physician reviews have an ability to help patients obtain quality care. They seem to serve best as a data-driven feedback loop for providers to train them to behave in accordance with standardized quality care processes. For payers, they serve to validate brands and the perceived value of care.
Recently, I completed a series of patient surveys of my own to determine how influential physician rankings are to persons seeking a medical provider. I asked patients in either crisis or maintenance, persons who recently had a need to find a provider for a serious medical condition and used the internet to find that provider. All patients surveyed, that’s 100%, said they placed no trust in the physician reviews.
Later, I asked a separate set – folks who needed only primary care services, and this set found physician ranking services to be praiseworthy for their offered convenience in locating the nearest provider with earliest availability. Rather than quality, users who appreciate physician ranking services value the convenience of healthcare, not quality.
Interestingly, again, none in this set cited a trust or belief in the rankings, just an appreciation for the convenience.
These findings do not invalidate the data value within physician ranking sites but rather disclose that these surveys are meaningful within the industry but not worth much to laypeople. They utterly fail and helping people achieve their claim: to find a good doctor.
As long as there is a separation of patients and payers, the notion that healthcare consumers will use the power of the crowd to define the best quality providers is an inherently false ideal. So I’m with the docs on this one. The weight placed on patient surveys should be 100-0.
Bob – you might want to be careful. I understand that CMS frowns upon “teaching to the exam” so to speak. Your script with patients dances a fine line.
On the other hand, checklists ensure nothing is missed. I read that portion of the post as quality assurance, rather than completely scripted. I agree that room for elucidation must exist, but I disagree that rote comments are wrong. Having a script can ensure the basics are told to all, while leaving the door open to discussion over and above that level.
A component of our hospital’s Physician Leadership Institute is completion of a team project. Last semester I was a little surprised when four participants (private-practice cardiologists and a family doc, none employed by our system) took on a project to improve the patient experience. Their Press Ganey scores placed them low nationally in percentile rank. Our program introduces evidence demonstrating the patient experience is correlated with clincial outcomes.
Within the framework of their project, they tested various approaches in the hospital. They landed on a set of simple, common-sense behaviors, which cleverly form the acronym RESPECT: Read chart/review labs before entering the room; make Eye contact; Sit down; connect with a Personal story; Explain next steps; Confirm patient understanding; and Time, as in “I have time for you…are there any questions or concerns?”
Their Press Ganey Scores are now well into the 90th percentile.
Bob, imagine if your dermatologist used RESPECT, how different your experience would have been. Far from “teaching to the exam”, these docs learned and applied new, authentic behaviors. Ratings can work. You can read their entire story in this month’s Press Ganey magazine.
Excellent post, Bob. Thanks.
David
Salem, OR
Kaiser Permanente has been thinking of patient satisfaction scores for a long time. One less advertised byproduct of this is that they are often incredibly liberal with providing opioid pills (hydrocodone and oxycodone) by the shovelful to patients with orthopedic problems large and small. Patients thus complain less in the short term, don’t come back to the facility as often, and I’m sure satisfaction scores are higher. But the fallout is tragic. More than a few of these patients have ended up in my addiction medicine practice due to iatrogenic opioid dependence, for example.
I think this focus on reviews and satisfaction scores may introduce perverse incentives to say “yes” when inappropriate (as in the antibiotics for URI example), or even worse to inappropriately mollify difficult patients with opioids or sedatives when that’s not actually in their long-term best interest.
I do not have faith that the reliability of assessment tools is anything but abysmal, whether it be an online review site or Medicare suboffice. Would be nice if it were otherwise, but it isn’t likely to become so, IMHO.
Exactly… its not a true evaluation of the hospitals performance. And with the “pain is what the patient says it is” obligation we have been held to… patient satisfaction will now insure us a bunch of opioid addicts and dependency because the doctors are no longer allowed to respond with no. Patient obligation and responsibility are gone. If you’re just a plain out dump at 300 lbs and are non compliant and return to our hospital system the hospital gets fined?! But don’t be insensitive and say your big ass is fat and you only want fast food medicine for the right now fix me… we get a bad score.
I just sent a letter to the head of the UCSF Medical Center, copies to Gov. Brown and the Senators, which you will find interesting. It is about a five-day stay at Parnassus notable for excellent surgical care and anesthesiology but which I found wanting in other regards. Please get hold of my letter and read it. Then please communicate with me. My experiences echo those of other patients I have talked with.
The problems I have identified are at Parnassus and are very serious.
I am Julius Comroe’s niece.
My notions of how to improve patient care do not, I believe, cost money. Rather, they depend upon improved coordination of services, improved communication, and improvements in nursing care using staff already present.
Incidentally, while I was in the hosptial I never saw a hosptialist – so far as I know.
Although there seem to be concerns on using ratings given by patients, I think it is something to be looked into. Most other ratings will look at the infrastructure of the hospital, the staff available and other things. But there are things beyond just those that makes the treatment complete. The level of openness of the doctors there, how eager are they to answer questions that family members may have, there are a lot of things. And I am happy to see that the hospital payments, at least a part of it, is dependent on these ratings.
The staff and the hospital as a whole will then hopefully go out of the way to make things as pleasant as possible for the patient. I do not mean the ‘hotel’ aspects in a hospital but the all things when considered in a holistic manner. And that is what I think should be the ultimate aim of a hospital.
The professional community, too, should understand this and move into the digital age wherein every shortfall will be pointed out and could hurt your reputation and business. Its already like that in a lot of professions and I don’t see a reason why it shouldn’t be so for an industry that is related to something so important to us, our biggest asset – our health.
Value based purchasing has transformed nursing into centering total care into discharge and nothing else. Meeting CHF, AMI, flu/ pneumonia prevention and discharge planning overrides everything we do at the bedside. And because of decreased income nurses are losing their hours and their bedside presence to really treat what you as a patient,to match budgeting needs, were there for. Also instead of providing you as the patient with better equipment and adequate staffing our facilities are putting in flat screen tvs because thats what satifies you. Thats really going to make you well? I agree that patients should be involved in their care but the popular opinion needs to remember why they sought a professional opinion to start with. Patients are involved in their care but now the healthcare providers aren’t. For everyone who pushed toward this way to go…. healthcare providers are losing their facets but remember the money you pay in as a good score goes directly into that CEOs pocket as they aren’t losing their time and salary. When you consider it doesn’t benefit you in a positive way at all does it really still seem patient based? Its all income based. Its not about you as the patient or us as the providers…. but the all mighty dollar. Had we staffed appropriately like we have banged our heads against the walls to get hospitals to do in the first place healthcare wouldn’t be driven to this. We lose millions to save a few hundred. And if the insurance companies would allow our physicians to practice as they are trained to do we wouldn’t waste healthcare dollars on loads of testing that really aren’t patient based… let’s be real either we are covering ourselves legally or meeting a standard to insure reimbursement by standards set by a coalition who are just as untrained as the patients we serve.
This is a very good thing that is happening, doctors and teachers should be reviewed by their patients and students respectively. This will help improve them improve themselves and their methods, and will also enable the institutions like hospitals, schools and colleges to judge them based on these reviews.