Chat GPT vs Real Docs

• Interesting results from at attempt to rate physician generated versus Chat GPT generated responses to patient questions posed on Reddit’s r/AskDocs:

Ayers JW, Poliak A, Dredze M, et al. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. Published online April 28, 2023. doi:10.1001/jamainternmed.2023.1838

“Results  Of the 195 questions and responses, evaluators preferred chatbot responses to physician responses in 78.6% (95% CI, 75.0%-81.8%) of the 585 evaluations. Mean (IQR) physician responses were significantly shorter than chatbot responses (52 [17-62] words vs 211 [168-245] words; t = 25.4; P < .001). Chatbot responses were rated of significantly higher quality than physician responses (t = 13.3; P < .001). The proportion of responses rated as good or very good quality (≥ 4), for instance, was higher for chatbot than physicians (chatbot: 78.5%, 95% CI, 72.3%-84.1%; physicians: 22.1%, 95% CI, 16.4%-28.2%;). This amounted to 3.6 times higher prevalence of good or very goodquality responses for the chatbot. Chatbot responses were also rated significantly more empathetic than physician responses (t = 18.9; P < .001). The proportion of responses rated empathetic or very empathetic (≥4) was higher for chatbot than for physicians (physicians: 4.6%, 95% CI, 2.1%-7.7%; chatbot: 45.1%, 95% CI, 38.5%-51.8%; physicians: 4.6%, 95% CI, 2.1%-7.7%). This amounted to 9.8 times higher prevalence of empatheticor very empathetic responses for the chatbot.

Conclusions  In this cross-sectional study, a chatbot generated quality and empathetic responses to patient questions posed in an online forum. Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit. Randomized trials could assess further if using AI assistants might improve responses, lower clinician burnout, and improve patient outcomes.”

The article itself has 5 sample questions with physician and Chatbot responses.  I’d agree that the Chatbot’s were better, both in content and tone.  Given the current possibility of hallucination, output should of course be reviewed before real clinical use, but boy, there will be pressure to use this…
2023-05-12T16:54:49-05:00May 12th, 2023|HomeRecommended|

The kids are not all right…

• David Wallace-Wells has another must-read piece in the NY Times: “It’s Not ‘Deaths of Despair.’ It’s Deaths of Children

Some excerpts:

“Americans are now dying younger on average than they used to, breaking from all global and historical patterns of predictable improvement. They are dying younger than in any peer countries, even accounting for the larger impact of the pandemic here. They are dying younger than in China, Cuba, the Czech Republic or Lebanon.

One in 25 American 5-year-olds now won’t live to see 40, a death rate about four times as high as in other wealthy nations.

Mortality is still increasing more quickly for those without a college degree, but as John Burn-Murdoch demonstrated vividly in The Financial Times, except for a few superrich Americans, individuals at every percentile of income are now dying sooner than their counterparts in Britain, for instance. For the poorer half of the country, simply being an American is equivalent to about four full years of life lost compared with the average Brit.

Black Americans, on average, can expect to live five fewer years than white Americans; Black American men have lower life expectancies than men in Rwanda, Laos and North Korea. White Americans, in turn, can expect to live seven fewer years than Asian Americans. Life expectancy in the Black Belt of the Deep South is as much as 20 years lower than it is north of the Mason-Dixon line and west of the Mississippi, according to the American Inequality Project.

The horror is that, as Burn-Murdoch memorably put it, in the average American kindergarten at least one child can expect to be buried by his or her parents.”

Wallace-Wells also refers to this viewpoint article published in JAMA Network by Woolf, Wolf, and Rivera.

2023-04-07T12:01:27-05:00April 7th, 2023|HomeRecommended|

Masks can make sense

• Tomas Pueyo, writing in his Uncharted Territories  substack blog takes a good hard look at the Cochrane Library’s meta analysis of the “Physical interventions to interrupt or reduce the spread of respiratory viruses” by Dooley et al. used by some (e.g. Bret Stephens in the NYT) to erroneously conclude that mask mandates are ineffective.  In reality, no such conclusion can be drawn from the Dooley review — the authors themselves state “The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions.” and “We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.”  Pueyo does a good job of analyzing why this review does not provide any substantive evidence that mask mandates are ineffective.

2023-02-28T17:33:18-05:00February 28th, 2023|HomeRecommended|

The Evil of the Murdoch Empire

• Jeremy Peters and Katie Robertson write in the NY Times about some of the noisome evidence produced in the Dominion defamation case. For me, the most heinous indictment in their story comes in this quote from Rupert Murdoch:

“On one occasion, as Mr. Murdoch watched Mr. Giuliani and Ms. Powell on television, he told Suzanne Scott, chief executive of Fox News Media, “Terrible stuff damaging everybody, I fear.””

Murdoch knew the damage he was doing to people and to this country and yet persisted, all in the interest of $$. Read the story, it says it all about the profit driven business plan of his media empire, one built on lies and fear-stoking.

2023-02-16T19:48:08-05:00February 16th, 2023|HomeRecommended|

How much do state policies influence life expectancy?

• Dr F. Perry Wilson of the Yale School of Medicine writes on Medscape about this interesting article from Plos One: “U.S. state policy contexts and mortality of working-age adults” by Montez, Mohri, Monnat, et al.  Dr. Wilson thinks that average education level may be the primary factor for the life expectancy discrepancy between states rather than state policies, but the authors in their abstract suggest that state policies may have a large impact:

“Simulations indicate that changing all policy domains in all states to a fully liberal orientation might have saved 171,030 lives in 2019, while changing them to a fully conservative orientation might have cost 217,635 lives.”

and in their discussion:

“On average, Americans die younger than their peers in most other high-income countries. In a 2013 U.S. survey, 85% of adult respondents indicated that their ideal life span was 79 years or older, yet U.S. life tables predicted that only 60% of people born that year could expect to survive to age 79 [40, 41]. Our findings, which examine working-age deaths among adults ages 25–64 years, suggest state policies–specifically, their left/right lean–may be a contributing factor and provide new insights into potential strategies to reduce working-age mortality.”

2023-02-02T19:23:22-05:00February 2nd, 2023|HomeRecommended|
Go to Top