You may read this post as the continuation of the one I made last year. Evaluate the risk caused by an action by comparing it with situations without that action. That is the core of the risk-benefit trade-off in decision-making. A third factor is missing in the equation, namely, the cost.
A new study published in The Lancet is the basis for this post. The report compiles the incidents of myocarditis and pericarditis, two well-known side effects linked to the mRNA vaccines against COVID-19. The data covered four health claim databases in the US and more than 15 million individuals.
The results
First, the overall summary: the data from four Data Partners (DP) indicate 411 events out of the 15 million studied who received the vaccine. Details of what is provided by each of the DPs are,
Data Partner (DP)
Total vaccinated
Total Observed myocarditis or pericarditis events (O)
Expected events (E) (based on 2019)
O/E
DP1
6,245,406
154
N/A
–
DP2
2,169,398
64
24.96
2.56
DP3
3,573,097
94
40.08
2.35
DP4
3,160,468
99
44.61
2.22
I don’t think you will demand a chi-squared test to get convinced that the two mRNA vaccines have an adverse effect on heart health. Age-wise split of the data gives further insights into the story.
Age-group
Observed Events
Total vaccinated
Incident Rate (per 100,000)
Expected Rate (per 100,000)
18-25
153
1,972,410
7.76
0.99
26-35
62
2,587,814
2.40
0.95
36-45
63
3,226,022
1.95
1.11
46-55
62
3,597,292
1.72
1.3
56-64
71
3,764,831
1.89
1.63
The relative risk is much higher for younger – 18 to 35 – age groups. But the absolute risk of the event is still in the single digits per hundred thousand. And this is where we should look at the risk-benefit-cost trade-off of decision-making.
The risk
First and foremost, don’t assume all those 411 individuals died from myocarditis or pericarditis; > 99% recover. To know that, you need to read another study published in December 2021 that reported the total number of deaths to just 8! So, there is a risk, but the absolute value is low. The awareness of the risk should alert the recipients that any discomfort after the vaccination warrants a medical checkup.
The benefit
It would be a crime to forget the unimaginable calamity that disease has brought to the US, with more than a million people dying from it. A significant portion of those deaths happened prior to the introduction of the vaccines, and even after, the casualties were disproportionately harder on the unvaccinated vs the vaccinated.
The cost
At least, in this case, the cost is a non-factor. Vaccine price, be it one dollar or 10 dollars, is way lower than the cost of the alternate choices, buying medicines, hospitalisation or death.
Managing trade-off
Different countries manage this trade-off differently. Since the risk of complications due to COVID-19 is much lower for children and the youth, some allocate a lower priority to the younger age groups or assign a different vaccine. However, it is recognised that avoiding their vaccination altogether, due to their low-risk status, is also not an answer to the problem. It can elevate the prevalence of illness in the system and jeopardise the elders with extra exposure to the virus.
References
Risk of myocarditis and pericarditis after the COVID-19 mRNA vaccination in the USA: The Lancet
We have already seen how the excess death rates (deaths per 100,000 population) due to covid distributed. The 25th percentile stands at 130 and 75th at 423 (as of 31st December 2021). The statistics of death rates is represented using a box plot.
The global distribution of excess death is sketched below:
Case of missed opportunity?
With all the support from hindsight knowledge, let us explore how much of these deaths could have been avoided (perhaps in the next pandemic!). Start with the top performers (the countries in green). These are true outliers and let us not fancy replicating their model. Australia, Newzealand, China, Singapore, Brunei are countries that opted for zero-covid policies, at least until a significant portion of their population received vaccines. They have closed down the countries and regions for the entire 2020 and the majority of 2021.
Bolivia tops the list in terms of excess deaths per million population, at 1376. The numbers have been bad from the beginning, and inadequate restriction measures, thanks to the chaotic political establishment, after the ouster of then-president, Evo Morales, did not help its course. Even today, Bolivia is far behind in vaccination rates.
While the exact reasons why Bulgaria is second in the global death charts is not known, I suppose it was not a coincidence that the country was the least vaccinated in Europe – just 27% by December 2021. For Peru, for instance, the story was poverty, lack of medical supplies, and oxygen. Delta variant and slow vaccinations are cited as the major reasons for the death toll in Russia.
The magenta counties
Brazil may be the model case of what not to do in a pandemic. The pandemic response was lax, and most of the deaths had happened in the first two waves, before the large-scale vaccination programs.
The US is an intriguing example. On the one hand, one can argue that the death rate of around 300 per 100,000 is the limit of what this disease can do with moderate barriers of disease control and a reasonable rate of vaccination. But the question will remain why the country can’t do what its neighbour Canada had managed (115). Spain too belongs to this category and is one of the countries that got battered in the first wave. The reason: no real preparedness as one of the earliest countries (after China and Italy) to hit the virus.
Final word for the country that topped the list of excess death – with about 4 million! India started with one of the most stringent covid measures in the world (shut down of March-May 2020). The country could not cope with the tides of the two waves, one starting from June 2020 and then the delta of 2021, with decent vaccination levels were so far away.
Reference
Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21: The Lancet
The risk of dying due to Covid was something that we discussed in the past. We observed (in October last year) that the absolute risk of death from covid was about 0.2 – 0.3%. Note that this is not the case-fatality ratio but the chance to die from Covid in a population. These values come from countries that are known for robust death registration systems. Also, the nations that were fully cut off from the rest of the world (e.g. New Zealand, Australia, China) during the first few phases of the pandemic were not considered.
This week The Lancet has published, by far, the most expensive data analysis on excess mortality attributed to Covid 19. Excess mortality is the difference between the number of deaths (all-cause mortality) during the pandemic (observed or estimated) and those expected from the past trends. The data used in the study included all-cause mortality data from various databases (global, regional and country-level) and empirical assessments.
18 million deaths in 2 years
The study reports that 18 million people had lost their life due to Covid in the first two years of the pandemic. That is about three times the official figures. There are about 56 million deaths occur in a year. Therefore, 18 mln in two years represents about 16%.
You can see from the plot that the 100-600 (deaths per 100,000 population) band enclosed most of the countries. The most notable outlier is China which, as per reports, have taken extreme measures to control the disease from spreading. The global average death rate is ca. 290 (without including China).
Another way of expressing the statistics of death rates is using a box plot.
Reference
Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21: The Lancet
This post follows up on the one made in December. Covid’s latest variant, the omicron, was a storm that played havoc over the world. The actual calamity is not known yet, but there is a widespread feeling that it was milder.
The UK, one of the countries that captured and shared covid data from day 1, provides answers to some of those questions. Here is an update of what happened since last time. Note the deaths of the last few days are incomplete.
Critical parameters such as hospitalisations and deaths have been rescaled – both x and y axes – to coincide with the reported cases before the vaccine.
Ever since the pandemic started in early 2020, one thing that polarised society was the severity of covid19. On one extreme were people who panicked over getting infected, and on the other were people who considered it as another spell of flu. What is the truth? Now that we have loads of data, it should be easier to find it out.
What is risk?
There are multiple definitions for the word risk. One of them, more technical, we have seen earlier. It is the product of the likelihood of something to happen and the consequence. The second one is from the oxford learner’s dictionary. The possibility of something bad happening at some time in the future; a situation that could be dangerous or have a bad result.
Who was right?
At the moment, both parties had reasons to believe in what they thought – it was risky to some and not to others. In other words, the risk was not the same for everybody. Look at the wealth of data collected by the CDC on cases in the US.
Age group
Total
18-29
40-49
50-64
65-74
75-84
85+
% in population
100
16.4
12.3
19.2
9.6
4.9
2
% infected
100
21.7
14.4
18.5
6.7
3.3
1.7
% died
100
0.8
4
17.6
22.1
26
27.7
population (estimated)
330
54.1
40.6
63.4
31.7
16.2
6.6
no of infected (mln) (estimated)
70
15.2
10.1
13
4.7
2.3
1.2
no of deaths (estimated)
0.85
0.007
0.034
0.15
0.19
0.22
0.23
infection rate (%)
21.2
28.1
24.8
20.4
14.8
14.3
18
death rate! (%)
0.26
0.01
0.09
0.25
0.6
1.3
3.3
death/death18-29 (any)
1
7
19
47
109
284
death/death18-29 (infected)
1
7.5
26
90
213
441
! The death rate is not the case fatality rate, it is the actual death rate in the population due to covid
Risks are not equal. Take some absolute numbers: the chance of someone dying of covid 19 (entire 2020-21) was about 0.25%. That doesn’t tell the whole story – for an 85-year-old, it is 3.3%. Another way is to calculate the chance of dying after getting infected. Overall it is ca. 1.2%, but for an 85+, it is ca. 20%!
Another type of risk estimate is relative to a younger age group. The relative risk is ca. 300 for an 85+ (any) of dying of covid, whereas once infected, the relative risk of dying is ca. 450.
You are wrong, it’s not flu
Society is connected. Calculating risks based on the least-risky age group is not the way to understand a contagious disease. Once a least-risky person comes home (or a care home), he has every chance of passing it to elders, whose risk was at least two orders of magnitude higher than the giver. For a modern society based on care-for-others, this is not a behaviour to be proud of.
Infectious diseases will come and go. Scientists will also find out cures for present and future pandemics. But, what is sure to remain untreated is human irrationality and ignorance of risks and asymmetry of life.
Asymmetry causes chaos in our brains; lack of data helplessness. Start with this news headline.
So what does this mean? The simple answer is – nothing! Because the percentage quoted in the headline (and the subsequent text) is the death of unvaccinated in the total deaths. It makes an implicit assumption that in the system, an unvaccinated can get serious illness in about 70 – 30 compared to vaccinated. That does not give the right picture about the vaccine.
Take a location with 1000 people, 100 deaths and create three scenarios. Scenario 1
Vaccinated
Unvaccinated
% of population
90%
10%
number of people
900
100
breakup of death
30%
70%
number of death
30
70
risk of dying
(30/900) = 0.033
(70/100) = 0.7
risk ratio
0.047
Scenario 1
Take the second scenario:
Vaccinated
Unvaccinated
% of population
50%
50%
number of people
500
500
breakup of death
30%
70%
number of death
30
70
risk of dying
(30/500) = 0.06
(70/500) = 0.14
risk ratio
0.43
Scenario 2
A third scenario
Vaccinated
Unvaccinated
% of population
10%
90%
number of people
100
900
breakup of death
30%
70%
number of death
30
70
risk of dying
(30/100) = 0.3
(70/900) = 0.077
risk ratio
3.9
Scenario 3
Discussion
Three scenarios using the same death-break up among vaccinated and unvaccinated tell three different stories. Scenario 1 shows a highly effective vaccine, the second is very modest, and the third is likely a substance to avoid! If you are not convinced, change the population from 1000 to any other number; you should get the same answer.
I agree journalists have a role in bringing information to the public. They also have a duty to provide data that enables the public to understand something. I doubt the news had any such intentions.
Finally
So, what is the big picture in Maharashtra? It’s difficult to say without details. But, assuming the number of deaths is more likely among adults, and its vaccination rates (at least one dose) are closer to 90%, the vaccine seems to protect as it promised.
Testing programs are not about machines but the people behind them.
We get into the calculations straight away. The equations that we made last time are:
Before we go further, let me show the output of 8 scenarios obtained by varying sensitivity and prevalence.
Case #
Sensitivity
Specificity
Prevalence
Chance of Disease for +ve (%)
Missed in 10000 tests
1
0.65
0.98
0.001
3
4
2
0.75
0.98
0.001
3.6
2.5
3
0.85
0.98
0.001
4
1.5
4
0.95
0.98
0.001
4.5
0.5
5
0.65
0.98
0.01
24
36
6
0.75
0.98
0.01
27
25
7
0.85
0.98
0.01
30
15
8
0.95
0.98
0.01
32
5
Chance of Disease for +ve = probability that a person is infected given her test result is positive. Missed in 10000 tests = the number of infected people showing negative results in every 10,000 tests.
Note that I fixed specificity in those calculations. The leading test methods of Covid19, RT-PCR and rapid Antigen are both known to have exceptionally low false-positive rates or specificities of close to 100%.
Now the results.
Before the Spread
It is when the prevalence of the disease was at 0.001 or 0.1%. While it is pretty disheartening to know that 95% of the people who tested positive and isolated did not have the disease, you can argue that it was a small sacrifice one did for society! The scenarios of low prevalence also seem to offer a comparative advantage for carrying out random tests using more expensive higher sensitivity tests. Those are also occasions of extensive quarantine rules for the incoming crowd.
After the Spread
Once the disease has displayed its monstrous feat in the community, the focus must change from prevention to mitigation. The priority of the public health system shifts to providing quality care to the infected people, and the removal of highly infectious people comes next. Devoting more efforts to testing a large population using time-consuming and expensive methods is no more practical for medical staff, who are now required at the patient care. And by now, even the highest accurate test throws more infected people into the population than the least sensitive method when the infection rate was a tenth.
Working Smart
A community spread also rings the time to switch the mode of operation. The problem is massive, and the resources are limited. An ideal situation to intervene and innovate. But first, we need to understand the root cause of the varied sensitivity and estimate the risk of leaving out the false negative.
Reason for Low Sensitivity
The sensitivity of Covid tests is spread all over the place – from 40% to 100%. It is true for RT-PCR, even truer for rapid (antigen) tests. The reasons for an ultimate false-negative test may lie with a lower viral load of the infected person, the improper sample (swab) collection, the poor quality of the kit used, inadequate extraction of the sample at the laboratory, a substandard detector of the instrument, or all of them. You can add them up, but in the end, what matters is the concentration of viral particles in the detection chamber.
Both techniques require a minimum concentration of viral particles in the test solution. Imagine a sample that contains lower than the critical concentration. RT PCR manages this shortfall by amplifying the material in the lab, cycle by cycle, each doubling the count. That defines the cycle threshold (CT) as the number of amplification cycles required for the fluorescent signal to cross the detection threshold.
Suppose the solution requires a million particles per ml of the solution (that appears in front of the fluorescent detector), and you get there by running the cycle 21 times. You get a signal, you confirm positive and report CT = 21. If the concentration at that moment was just 100, you don’t get a response, and you continue the amplification step until you reach CT = 35 (100 x 2(35 – 21) – 2 to the power 14 – is > 1 million). The machine suddenly detects, and you report a positive at CT = 35. However, this process can’t go forever; depending on the protocols, the CT has a cut-off of 35 to 40.
On the other hand, Antigen tests detect the presence of viral protein, and it has no means to amplify the quantity. After all, it is a quick point of care test. A direct comparison with the PCR family does not make much sense, as the two techniques work on different principles. But reports suggest sensitivities of > 90% for antigen tests for CT = 28 and lower. You can spare a thought at the irony that an Antigen test is sensitive to detect the presence of the virus that the PCR machine would have taken 28 rounds of amplification. But that is not the point. If you have the facility to amplify, why not use it.
The Risk of Leaving out the Infected
It is a subject of immense debate. Some scientists argue that the objectives of the testing program should be to detect and isolate the infectious and not every infected. While this makes sense in principle, there is a vital flaw in the argument. There is an underlying assumption that the person with too few counts to detect is always on the right side of the infection timeline – in the post-infectious phase. In reality, the person who got the negative test in a rapid screening can also be in the incubation period and becomes infectious in a few days. They point to the shape of the infection curve, which is skewed to the right, or fewer days to incubate to sizeable viral quantity and more time on the right. Another suggestion is to test more frequently so that the person who missed due to a lower count comes back for the test a day or two later and then caught.
How to Increase Sensitivity
There are a bunch of activities the system can do. The first in the list is to tighten the quality control or prevent all the loss mechanisms from the time of sampling till detection. That is training and procedures. The second is to change the strategy from analytical regime to clinical – from random screening to targetted testing. For example, if the qualified medical professional identifies patients with flu-like symptoms, the probability of catching a high-concentrated sample increases. Once that sample goes to the testing device for the antigen, you either find the suspect (covid) or not (flu), but it was not due to any lack of virus from the swab. If the health practitioner still suspects, she may recommend an RT PCR, but no more a random decision.
In Summary
We are in the middle of a pandemic. The old ways of prevention are no more practical. Covid diagnostics started as a clinical challenge, but somewhere along the journey, that shifted more to analytics. While test-kit manufacturers, laboratories, data scientists and the public are all valuable players to maximise the output, the lead must go back to trained medical professionals. A triage system, based on experiences to identify symptoms and suggested follow up actions, is a strategy worth the effort to stop this deluge of cases.
We have seen the definitions. We will see their applications in diagnosis. As we have seen, both Sensitivity and Specificity are probabilities, and the diagnostic process’s job is to bring certainty to the presence of a disease from the data. And the tool we use is Bayes’ theorem. So let’s get started.
We tailor the Bayes’ theorem for our screening test. First, the chance of being infected after the person was diagnosed with a positive test. Epidemiologists call it positive predictive value or, in our language, the posterior probability.
Positive Predictive Value (PPV)
Looking at the equation carefully, we can see the following. P(+|Inf) is the true positive or the sensitivity, and P(+|NoInf) is the false positive or (1 – Specificity). It leaves two unknown variables – P(Inf) and P(NoInf). P(Inf) is the prevalence of the disease in the community, and P(NoInf) is 1 – P(Inf).
And we’re done! Let’s apply the equation for a person who tested COVID-19 positive as part of a random sampling campaign in a city with a population of 100,000 and 100 ill people. The word random is a valuable description to remember; you will see the reason in a future post. Assume a sensitivity of 85% (yes, for your RT-PCR!) and a specificity of 98%.
Chance of Infection = 0.85 x 0.001 /(0.85 x 0.001 + 0.02 x 0.999) = 0.04. The instrument was of good quality, the health worker was skilled, and the system was honest (three deadly assumptions to make), yet she had only a 4% chance of infection.
Negative Predictive Value (NPV)
Now, quickly jump to the opposite: what is the chance someone who got tested negative, escapes the diagnostic web of the community?
There is a 99.98% certainty of no illness or a 0.02% chance of accidentally escaping the realm of the health protocol.
What These Mean
In the first example (PPV), a 4% chance of infection means relief to the person eventually, but there is a pain to do the mandatory ‘insolation’ as the system treats her as an infected.
The second one (NPV) is the opposite; for the individual, 0.02% is low; therefore, a test with medium sensitivity is quite acceptable. For the system, which wants to trace and isolate every single infected person, this means, that for every 10,000 people sampled randomly, there is a chance to send out two infected individuals into the society.
We have made a set of assumptions regarding sensitivity, specificity and prevalence. And the output is related to those. We will discuss the reasons behind these assumptions, the cost-risk-value tradeoffs, and the tricks to manage traps of diagnostics. But next time. Ciao.
Screening tests such as PCR are typically employed to test the likelihood of microbial pathogens in the body. Test results are estimates of probability and are evaluated by trained medical professionals to confirm the illness or to recommend any follow-up actions. Two terms that we have extensively used in the last two years have been the sensitivity and specificity of covid tests.
Sensitivity: Positive Among Infected, P(+|Inf)
Sensitivity is a conditional probability. It is not the ability of the machine to pick ill people from the population, although it could be related. But it is:
A test’s ability to correctly identify from a group of people who are infected.
P(+|Inf) – the probability of getting a positive result given the person was infected.
A test has a sensitivity of 0.8 (80%) if it can correctly identify 80% of people who have the disease. However, it wrongly assigns 20% with negative results.
Specificity: Negative Among Healthy, P(-|NoInf)
A test’s ability to correctly identify from a group of people who are not infected.
P(-|NoInf) – the probability of getting a negative result given the person was not infected.
A test with 90% specificity correctly identifies 90% of the healthy and wrongly gives out positive results to the rest 10%.
Final Remarks
We’ll stop here but will continue in another post. Sensitivity = P(+|Inf) = 1 – P(-|Inf). If you are infected, a test can either give a positive or a negative result (mutually exclusive probabilities). In other words, you are either true positive or false negative.
Specificity = P(-|NoInf) = 1 – P(+|NoInf). If you are healthy, a test can either give a negative or a positive test result – a true negative or a false positive.
Does a positive result from the screening test prove the person is infected? No, you need to know the prevalence to proceed further. We’ll see why we developed these equations and how we could use them to evaluate test results correctly.
Let me start with a disclaimer: this is purely for demonstration purposes. The numbers used in the following analysis should not be viewed as an accurate description of the complex biological processes in the body.
In an earlier post explaining vaccination, I had mentioned the law of mass action. It is also called chemical kinetics. For a chemist, everything is a reaction, and solving kinetic equations are the way of understanding the world around her.
Equations of life
Molecules react to form products. Consider the following hypothetical reactions.
(1)
(2)
(3)
(4)
V represents virus, A for antibody, C for cells and P for blood plasma.
As per the law of mass action, the speed of a reaction is related to its rate constant and concentrations of ingredients. The four items above translate to a set of differential equations,
(5)
(6)
(7)
What do these equations mean?
All three equations have a rate (speed) term on the left and a set of additions (production) and subtractions (consumption) on the right.
The speed of each reaction is related to the concentrations of the constituents.
If a reaction rate constant increases, the speed of the reaction increases.
The production rate of antibodies (from blood plasma) is assumed constant.
Let us solve these three differential equations simultaneously. I used the R package ‘deSolve’ to carry out that job.
Case 1: A person in a risky group and no vaccination
Used the following set of (arbitrary) numbers: k1 = 0.45, k2 = 0.05, k3 = 0.01, k4 = 0.01. Intial concentrations (time = 0) Ca = 0, Cc = 100, Cv = 1.
You can see that the person is in real danger as all her cells have been attacked by the virus that multiplied exponentially.
Case 2: A person with healthy antibody production and no vaccination
Now, use exactly the same input, but the antibody production rate constant k2 is 4 x: k1 = 0.45, k2 = 0.2, k3 = 0.01, k4 = 0.01.
The initial growth of the virus was curbed down pretty fast by the antibodies and the person survived.
Case 3: Risky group and vaccination
The parameters are the same as in case 1, but 5 units of antibodies are available at time zero (from vaccination).
Case 4: Risky group, vaccination, double viral load
Same as case 3, but the initial viral concentration doubled – from 1 to 2.
Case 5: Risky group, booster vaccination, double viral load
Same as case 4, but the antibodies from vaccination was double, or at ten units.
Case 6: No vaccination and double viral load
This case was created to show the speed at which the virus took control over the body. The parameters are exactly the same as case 1, but the initial virus load increased to 2 from 1.
In summary
These are simplistic ways of picturing what dynamics are going on in our body once a virus comes in. Treatments (mathematical) like these can also expand our imagination to newer ways of managing the illness. Say, can we find a way to reduce the rate constant k1 (viral replication)? Antiviral drugs such as ‘molnupiravir’ are expected to do precisely this.
Mechanism of molnupiravir-induced SARS-CoV-2 mutagenesis: Nature
