How to understand medical decisions

If a diagnosis is uncertain what are all the other possible  diagnoses and their probabilities and how were the probabilities derived?

If the diagnosis is uncertain the answer to the above question should contain a list of diagnoses and the estimated frequency with which each one will be confirmed.  If these frequencies add up to more than 90% there is a probability of more than 0.9 that the diagnosis will be one of these.  


Meaning of probability

A probability of 0.9 would be the degree of certainty experienced that a patient has diabetes if there were 10 people in a waiting room and 9 had diabetes and if you met one of them, only knowing that the person was one of the 10 in the room.  There are many ways of estimating a probability, but whatever the method, the estimated probability (e.g. of 0.9) can be interpreted AS IF it had been arrived at in the above hypothetical way.


Lists of possibilities

Each diagnosis might be uncertain because the features that are present could be explained by a number of different diagnostic possibilities (called the ‘differential diagnoses’ of those features).  These lists of possibilities are based on past experience, received wisdom passed down from one doctor to another and written in textbooks and (not often enough) carefully recorded experience of similar situations.  The estimated proportion of times each of these differential diagnoses will be confirmed later can be regarded as its ‘probability’.  


Guessed or observed probabilities

A doctor can arrive at probability estimates in a number of ways.  They can be guesses of the frequency of correct predictions based on current theories about the diagnosis.  They can be based on the doctor’s personal experience (which is usually based on memory that can be mistaken).  They can be based on the doctor’s carefully recorded outcomes of groups of patients with identical features (such information is very difficult to get without research funding and support).  They can be based on guesses based on observations made by someone else on a group of patients that are similar but not identical and described in research publications.



Some doctors keep a mental tally of how often their predictions turn out to be correct, irrespective of what the prediction is about.  For example, if they are correct 95 out of 100 times after they had estimated that the probability was 0.95 or 95% (irrespective of what the prediction is about) then their overall judgment is reasonable.  However, if they were only correct 70/100 times for all their estimated probabilities of 0.95, they should re-calibrate their judgment by changing their estimated probabilities for that mental feeling of certainty from 95% down to 70%.  For doctors who keep doing this, their probability estimates would be reasonable overall. They may also check their probabilities for different predictions (e.g. cardiac conditions or a particular cardiac condtion) to see if some area of knowledge is letting them down and dragging down their overal judgement.


Published observations

It is not possible for all doctors to build up enough carefully recorded personal experience that can provide reliable probabilities for looking after future patients.  Such experience is usually obtained by funding clinical research by specially trained doctors that other doctors can read.  When they read about each observation performed by someone else they have to ask an important question: “If I repeated this observation in my own setting, what is the probability that I would get a similar result?”  This can be termed the ‘probability of replication’.  It is fundamental to science.



The probability of replicating a result between two ranges (e.g. that someone else’s observation of 70/100 will be replicated by a reader within a range of 60/100 and 80/100) can be estimated by first estimating that the probability of NOT replicating the result within this range for different reasons.  Thus if the probability of non-replication due to chance is low (e.g. if the number of observations had been high e.g. 700/1000), if the probability of non-replication because of other contrary observations by others is low (e.g. there were few other observations different to 70%), if the probability of non-replication because of patients, setting and methods for the published observations were different to the reader’s is low, if the probability on non-replication because of inaccurate reporting, dishonesty, etc is low, then the probability of replication by the reader will be high.


Facts or evidence

The principle of replication applies to the results of passive observations on populations and also to the results of clinical trials where an active treatment has been given to some patients.  It is central to the idea of ‘evidence based medicine’, which aims to base all estimates of probabilities on research that involves careful observations.  These observations are ‘facts’.  They become ‘evidence’ when they are used to make predictions.


'General' and 'particular evidence'

There is another type of evidence used in ‘evidence-based medicine’.  This is based on the ‘facts’ obtained from the individual patient as opposed to populations of patients described in a publication.  These other facts are a patient's findings – symptoms, physical signs and test results.  When these findings are used to make predictions about a diagnosis or outcome, they too become ‘evidence’.  Because the facts are obtained from a ‘particular’ individual, they can be described as ‘particular’ evidence.  This is contrast the 'general evidence' obtained from a ‘general’ population.  General facts and evidence based on populations are examples of general propositions.  Particular facts and evidence based on individual patients are examples of particular propositions.


© Huw Llewelyn 2016