This Community is Archived.

This community is no longer active as of December 2018. Thanks to those who posted here and made this information available to others visiting the site.

0 Recommendations

Keynote Felix Greaves on using online reviews to improve patient experience

By Rebecca Weintraub, MD | 12 Dec, 2014

We are pleased to welcome Felix Greaves, honorary senior clinical lecturer in the Department of Primary Care and Public Health at Imperial College London and a public health doctor, as a Keynote Speaker for this Breakthrough Opportunities event with the Commonwealth Fund.

Felix's research interests are in measuring quality and safety in healthcare systems. He is currently evaluating 'Tripadvisor' models of online feedback in healthcare in the UK, and whether patients' descriptions of their care online can be used to measure health system performance.
He was previously clinical adviser to the Chief Medical Officer at the Department of Health, where he worked on developing national quality and safety policy. He also worked for the World Health Organization’s Patient Safety Programme, where he managed their project on improving patient safety education in medical schools and technology for patient safety.

In this video interview, Felix speaks to the potential of online ratings and reviews as well as social media commentary to determine the quality of care, as well as the need for health care systems to pay attention to this information to gain insight and improve.

Felix poses a challenge to us all: How can we use online patient commentary to drive service improvement?

We look forward to hearing your thoughts on Felix’s video below, and encourage you to submit ideas sparked by his closing question:

(Note for those viewing this update via email, please click through to view Felix’s video on


Zvi Frankel Replied at 2:46 PM, 12 Dec 2014

Thank you Felix for your great presentation. In your opinion, is it possible to precisely draw conclusions from data collected from unverifiable online feedback sources (which as you noted in your remarks, include the potential for "self-solicited information" and of "gaming"), without simultaneously having the perspective acquired by collecting and comparing actual properly risk-adjusted outcomes data (which doesn't have the potential for "self-solicited information" and of "gaming")? Thanks again.

Felix Greaves Keynote Speaker Replied at 2:08 PM, 14 Dec 2014

Hi Zvi, Thanks for your question.
In my view, the two methods of collecting patient feedback - solicited vs unsolicited - should be viewed as complementary, with neither as replacement for the other. They are two different ways to 'biopsy the patient voice', and sample different but overlapping groups. And both have their own advantages and disadvantages. Structured, solicited, and risk adjusted data will be better for making comparisons between services - but often perfect data is not available, complicated to interpret, and expensive to collect. Unsolicited data allows anyone to comment, and is often trusted by the public, and is cheap to collect. - however, it is not representative and can be gamed. If you have both, use both. If only one, I'd suggest using it, but with an understanding of the limitations of that perspective.

You ask if you can 'precisely draw conclusions'. I'd say not precisely, but it can give you new information that is potentially useful and relevant.


Zvi Frankel Replied at 5:37 PM, 14 Dec 2014

Hi Felix, Thanks for your response. I agree that the two sources of data could be viewed as complementary, utilized in harmony to build a comprehensive picture to empower the consumer, while protecting physicians and patients alike (from "gaming," etc.).
However, when it comes to outcomes data, proper risk adjustment is recognized as an essential safeguarding tool to utilize as a way to certify that published outcomes data is beneficial, and not detrimental to patients and physicians (e.g. to avoid the occurrence of cherry picking, etc.). For that reason, there is a widespread accepted consensus that outcomes data which is not risk adjusted could be quite counterproductive. In your opinion, what is the equivalent safeguarding tool, which could be used in the context of online feedback sources data, to ensure that the data is likewise fair, beneficial, and not detrimental?


Felix Greaves Keynote Speaker Replied at 12:54 PM, 15 Dec 2014

Thanks Zvi,
I think that there are some technical safeguards to prevent gaming and other manipulation of the data - such as preventing repetitive patterns of review from one source or algorithmic detection of suspicious patterns - that can be used to remove the most flagrant abuses from the system. However beyond that, I think this is data that is very hard to adjust appropriately - and we may have to accept that. The data on Tripadvisor are not risk adjusted depending on the type of client the hotel serves - yet the public find the information relevant and meaningful. I appreciate that a comparison between the hospitality sector and the practice of medicine has many limitations, but very few patients make choices about hospitals on the basis of risk adjusted data, which they find complex, abstract and lacking much empathetic connection with the user.

I think there is a place for this unfiltered data alongside the other sources. In England, we had a substantial failure of hospital quality that took far too long to be discovered - at a hospital organisation called Mid Staffordshire. The substantial national review of the event afterwards found that although the risk adjusted data showed poor performance, it was consistently brushed aside as being 'poor data' or with the traditional 'our patients are different' story. In the end, it was the stories of the individual patients - too often ignored - that drove realisation of the extent of the problem and forced a formal inquiry. Stories, and unfiltered patient ratings, have a role alongside the other data driven approaches.

On a more practical note, patients are going to do this whatever happens. You just need to look at companies like Uber and AirBnB to see that peer feedback is becoming normal - and to an extent trusted. We should use these changing societal norms and behaviours to our advantage.

Zvi Frankel Replied at 2:16 PM, 15 Dec 2014

Thanks Felix, I appreciate the great points you raise. As you allude to in your comments, I think there is indeed a fundamental difference between using online public feedback to decide which hotel to visit, versus deciding which hospital to undergo a life and death medical procedure. Would you agree that in general, the number one priority patients have when undergoing a serious medical procedure, is that the procedure will be successful? For example, if a patient is given a choice of being treated by a less than friendly surgeon with a 0% risk adjusted mortality rate, versus a surgeon with the best possible bed-side manner, albeit with a 10% risk adjusted mortality rate, which do you think would be the overwhelmingly more popular pick? However, if risk adjusted outcomes data is not publicly available, do you think that there is reason for legitimate concern that online public feedback may likely reflect variables other than outcomes, and thus mislead patients who base their decision on such data (and thus potentially harm patients, by leading them to physicians with inferior outcomes, and at the same time potentially harm the reputation of surgeons who may be presented as inferior, when they in reality have the best outcomes)? I agree that a "Tripadvisor" model medical website would be an excellent resource, but I wonder if it would have the opposite effect if not combined with actual risk adjusted outcomes data. I think it's essential for consumers to be empowered with all the facts, to enable them to make fully informed health decisions. For example, if one could read the public feedback regarding a surgeon, and then click for the actual risk adjusted outcomes data--wouldn't that preserve the wonderful advantages of unsolicited feedback data, while avoiding very serious unintended ramifications?

Also, could please elaborate regarding what you wrote that, "...very few patients make choices about hospitals on the basis of risk adjusted data, which they find complex, abstract and lacking much empathetic connection with the user..." Do you think that patients would not overwhelmingly embrace the access to the above described mortality outcomes data, if explained alongside public feedback in a clear user friendly format?

This Community is Archived.

This community is no longer active as of December 2018. Thanks to those who posted here and made this information available to others visiting the site.