Practitioners are experts. They could be medical practitioners, domain experts, lawyers and judges, leaders of organisations, sports persons-turned-pundits, to name a few. A lot of decision making rests on their shoulders, and the tool they often employ is experience. And experience is a double-edged sword! On the one hand, it makes them the most suitable people for the job, but on the other hand, they tend to ignore quantitative inference and rely on personal experience instead.
JAMA Internal Medicine collected responses from 723 practitioners from outpatient clinics in the US and published a paper in April 2021. The study aimed to estimate the understanding of risks and clinical decisions taken by medical practitioners. They included physicians and nurse practitioners. They were given a questionnaire to fill in the pretest and post-test probabilities of a set of illnesses. The requested post-test estimates included those after positive tests and negative tests.
The survey had five questions – four containing clinical scenarios (pneumonia, breast cancer, cardiac ischemia and UTI) and one hypothetical testing situation (a disease with 0.1% prevalence and test with 100% sensitivity and 95% specificity). The scientific evidence and the median responses are tabulated below:
| Clinical Scenario | Scientific evidence | Estimate Resident physician | Estimate attending physician | Estimate Nurse practitioner |
| Pneumonia | ||||
| pretest probability | 25-42 | 80 | 85 | 80 |
| post-test after +ve test | 46-65 | 95 | 95 | 95 |
| post-test after – ve test | 10-19 | 60 | 50 | 50 |
| breast cancer | ||||
| pretest probability | 0.2 – 0.3 | 5 | 2 | 10 |
| post-test after +ve test | 3 – 9 | 60 | 50 | 60 |
| post-test after – ve test | < 0.05 | 5 | 1 | 10 |
| cardiac ischemia | ||||
| pretest probability | 1-4.4 | 10 | 5 | 15 |
| post-test after +ve test | 2-11 | 75 | 60 | 90 |
| post-test after – ve test | 0.43-2.5 | 5 | 5 | 10 |
| UTI | ||||
| pretest probability | 0-1 | 25 | 20 | 30 |
| post-test after +ve test | 0-8.3 | 77.5 | 90 | 90 |
| post-test after – ve test | 0-0.11 | 5 | 5 | 5 |
| Hypothetical Scenario | ||||
| post-test after +ve test | 2 | 95 | 95 | 95 |
| post-test after – ve test | 0 | 2 | 5 | 5 |
Those unheard are …
Before pointing fingers at the medical practitioners: you get this data because someone cared to measure, the specialists were happy to cooperate, and the Medical Association had the courage and insight to publish it. And the ultimate objective is quality improvement.
At the same time, the survey results suggest the lack of awareness of the element of probability in clinical practice and call for greater urgency to focus on scientific, evidence-based medical practice.
Morgan et al., JAMA Intern Med. 2021;181(6):747-755
