Representativeness Heuristics

Heuristics are mental shortcuts or straightforward rules of thumb, often developed from past experiences, used to make quick decisions. While it helps enormously to cut down time and effort to make decisions – decisions are taxing to the brain – occasionally, it can also lead to troubles. For example, a popular heuristic, the availability bias, makes us think that we live in an era of violence more than ever before, thanks to the day-to-day images we see in the media.

Here, we look at another one – the representativeness heuristics. The best way to describe it is:
“If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck”.

In representativeness heuristics, when compelled to make a decision, one compares herself with a prototype (or stereotype) of an event or behaviour she already has in mind.

In the famous Linda’s Problem, the image of a girl who participates in a demonstration drives us to tag her as a feminist.

A known example with more serious implications is racial profiling. It is when the police search for a crime suspect or an airport security officer doing random checks disproportionately focus on blacks or people of colour.

Representativeness Heuristics Read More »

The Science We Trust

People lament about the dominance of beliefs and the reduction of scientific temperament in society. Unfortunately, it is a fact and can only be worse in future. And I want to argue that it can only be like that. Let’s look at a few reasons why achieving scientific character is a mission impossible.

It’s another religion

Unfortunately, it has to be.

Take the example of the discovery of gravitational waves in 2015. The number of people involved in the observation, which includes the setting hypothesis, the detection, and the mathematical modelling, could be about 1000. The rest of the world (1000 short of 7 billion) only gets the publication, which is already a heavily cut-down, readable version of the actual data.

Imagine a million people downloaded the paper.

As per an old report in physics today, the percentage of physics graduates (minimum decent training level in this field) was about 0.01. It suggests the inconvenient truth that 99.99% of people are already at a considerable disadvantage.

I.e., half all physics graduates and the rest others!

The people who understand the model (the specific mathematics behind the event) are even fewer and could be in the hundreds at best.

All the others – 6999 million out of 7000 – get the news from the media. And they must trust the report. A belief system is created but is not going to last like a religion, as we shall see soon.

What is Science

Most people know science through technology, the application of the former into products. To define it in one word: science is hypothesis testing. And most people are alien to it. It is probabilistic, conditional, and will/must update with time. Each of these contradicts the doctrines of religion.

Probabilistic thinkers meet the real people

Back to the gravitational waves: Movements on the ground, temperature changes in the instruments or numerous other known or unknown errors can all lead to artificial signals or noises. The importance of the results led to keeping a significance level for the rejection of the null hypothesis (that the observed signal is a noise) to be extremely low – one in a billion. If you recall, most of our ordinary life experiments that is one in 20!

The team investigating the gravitational waves published the findings (as real) only when they found the probability that it could happen by chance is one in a billion. Yet, they would only use the words such as ‘likely’, ‘probably’ or ‘mostly’, to respond to the public, who want ‘yes’ or ‘no’ as answers.

And they change with time

Science updates with new information. Remember the chaos during covid time? The understanding of the illness changed daily during the pandemic. The use of masks (to use or not to use) and modes of contagion (airborne vs liquid-borne), to name a few. While the changes of advice were perfectly understandable and acceptable for those scientists, it was causing confusion and anger to the 99.99%.

The Science We Trust Read More »

Happiness and PCA

Let’s do a principal component analysis of the underlying variables in the estimation of the Happiness Index. They are

Real GDP per capita
Social support
Healthy life expectancy
Freedom to make life choices
Generosity
Perceptions of corruption

The objective is to see how countries are clustered together in the PCA.

Happiness and PCA Read More »

PCA of NBA Players

Let’s now move to NBA. Following is the PCA biplot of the ESPN top 40 NBA players of the regular season 2022-23.

We can see a few things:
1) Damian Lillard and Steph Curry are in a cluster which is closer to the vector 3PM (three points made)
2) A few centres are closer to each other, and the vector BLKPG (blocks per game) is closer to them.
3) Jokic and Giannis are placed somewhere far away.
4) APG (assists per game) and TOPG (turnover per game) are similar contributions (negative) to the principal component 2. The leaders, Harden, Haliburton and Young, are closer to the APG vector.
5) Centres and power forwards dominate the right side of principal component 1, whereas the guards take the left.

We see 3PM and FG% (field goal percentages) diametrically opposite to each other, suggesting they are negatively correlated.

And, if you are wondering who they are:

The data are taken from the ESPN site using the following R code:

library(rvest)
nba_23 <- read_html("https://www.espn.com/nba/seasonleaders")
nba_23 <- nba_23 %>% html_table(fill = TRUE)

Followed by a few clean-up steps

nba_data <- as.data.frame(nba_23)
names(nba_data) <- nba_data[2,]
nba_data <- nba_data[-1:-2,]
index <- which(nba_data$PLAYER == "PLAYER")
nba_data <- nba_data[-index,]
nba_data <- nba_data %>% mutate_at(vars(GP, MPG, `FG%`, `FT%`, `3PM`, RPG, APG, STPG, BLKPG, TOPG, PTS), as.numeric)

References

2022-2023 NBA Season Leaders: ESPN

PCA of NBA Players Read More »

Principal Component Analysis Applied

Let’s apply what we learned in the ‘mtcars’ data. We use R to perform the calculations. We require two packages, ‘stats’ and ‘ggbiplot’, to do the job.

library(stats)
library(ggbiplot)

Start with the simplest first – two variables – mpg and disp

data("mtcars")
car_data <- mtcars
mtcars.pca <- prcomp(car_data[,c(1,3)], center = TRUE,scale. = TRUE)
ggbiplot(mtcars.pca,
  ellipse = TRUE,
  labels = rownames(car_data)
)

You can see a few clusters – things on the right, left and centre. You can also see two arrows, one corresponding to mpg and another to disp. It’s true we don’t need to do a PCA for two variables; a 2-D can do the job already.

You can already start interpreting the PCA plot. The Cadilac and Lincoln are closer to the disp line in the PCA plot, which is towards the northwest of the Displacement vs Mileage plot. On the other hand, Honda, Porche etc., are closer to the mpg axis.

mtcars.pca <- prcomp(car_data[,c(1,3, 6, 7)], center = TRUE,scale. = TRUE)
ggbiplot(mtcars.pca)
ggbiplot(mtcars.pca,
  ellipse = TRUE,
  labels = rownames(car_data)#,
)

Principal Component Analysis Applied Read More »

Principal Component Analysis – The Concept

Last time we saw the practical difficulty of analysing data from four or more measured variables. The demands a means of reducing the numbers to two so that it appears on a 2-D plot but gives the message we want – that similar candidates cluster together.

In other words, one must perform necessary mathematical manipulations to convert the parameters to a different set of variables (principal components), select the top two or the principal components, and plot them. All these happen without losing much of the information embedded inside it.

PCA is the technique of compressing data from a large set of measurements into a smaller number of independent (i.e., uncorrelated) variables that captures the core of the original data. Note that the principal component themselves are linear combinations of the original variables.

The first principal component, which becomes the X-axis, defines the direction of the maximum variation of data.

Principal Component Analysis – The Concept Read More »

Principal Component Analysis – Building the Case

Do you remember the “mtcars” dataset? It’s data collected from the 1974 Motor Trend US magazine and it comprises fuel consumption and ten aspects of automobile design and performance for 32 automobiles (1973–74 models). We’ll use it to explain the concept of principal component analysis or PCA.

If we measure only one aspect, we can present the data on a line plot:

You can see that Toyota Corolla, Fiat 128 etc., are similar to each other, and have relatively higher mileage values, whereas Cadillac Fleetwood and Lincoln Continental have lower.

If we measure two properties, we can present the data in a 2-D graph.

If we measure one more property, we would add one more axis to the graph for a 3-D plot. But what happens if we have four or more parameters? PCA can take four or more measurements and make a 2-D PCA plot.

Principal Component Analysis – Building the Case Read More »

Population Inflection

The news that India has overtaken China as the most populous country in the world sparked a flurry of debates in the public discourse. And, as usual, many of them instilling fear and aimed at demonising specific communities. But, as we have seen before, the data was not as bad as one would imagine.

And the reason is visible in the following plot. You may see an inflection point, denoting a change in growth rate (not absolute growth). The location of inflection is estimated using R with the help of the package “inflection”.

x = in_pop$Year[1:72]
y = in_pop$All[1:72]/1e6


plot(x,y,cex=0.3,pch=19, ylab = "Population in Millions", xlab = "Year", ylim = c(0, 1500), col = "blue", type ="l", lwd=3)
grid()

bb <- ese(x,y,0)
pese <- bb[,3]

abline(v=pese, col="red", lwd=2, lty=2)

And this will lead to an eventual peak and a further decline, as per projections.

In the following plot, you will see what happens to the different age groups. The under-25 (green) has already peaked, 25-65 (brown) will be in a couple of decades from now and the old (> 65, white) to stay flat by the end of this millennium.

India’s population growth will come to an end: Our World in Data

Population Inflection Read More »

The Lost Diamond of Bayes

Here is a problem that combines combinations with Bayes’s rule. A card is lost from the 52-card deck. Two cards are drawn from the deck and found to be both diamonds. What is the probability that the lost card is a diamond?

Let’s write down Bayes’ equation first.

P(L_D|2_D) = \frac{P(2_D|L_D)*P(L_D)}{P(2_D|L_D)*P(L_D) + P(2_D|L_{nD})*P(L_{nD})}

P(LD|2D) = The probability that the lost card is a diamond, given two diamonds are drawn.
P(2D|LD) = The probability of drawing two diamonds if the lost card is a diamond
P(LD) = The probability of losing a diamond.
P(2D|LnD) = The probability of drawing two diamonds if the lost card is not a diamond
P(LnD) = The probability of losing a card other than a diamond.

Evaluating each term,
As there are 13 diamonds in a pack of 52 cards, P(LD) is 13 in 52 (13/52 = 1/4), and P(LnD) is 52-13 in 52 (3/4).
P(2D|LD), or the probability of drawing two diamonds from a deck with a missing diamond, is 12C2 / 51C2 = 12 x 11 / (51 x 50).
P(2D|LnD), or the probability of drawing two diamonds from a deck with a missing non-diamond, is 13C2 / 51C2 = 13 x 12 / (51 x 50).

\\ P(L_D|2_D) = \frac{\frac{12*11}{51*50}*\frac{1}{4}}{\frac{12*11}{51*50}*\frac{1}{4} + \frac{13*12}{51*50}*\frac{3}{4}} \\ \\ \frac{12*11*(1/4)}{12*11*(1/4) + 13*12*(3/4)} = \frac{11}{50}

P(LD|2D) = 11/50 = 22%

The Lost Diamond of Bayes Read More »

Committee of Couples

From a group of five married couples, how many committees of four or five people can be formed if no two people on the committee may be married to each other?

4-member commitee

There are 5C4 ways to choose four couples. Then there are 2C1 ways to pick one person from each couple.

5C4 x 2C1 x 2C1 x 2C1 x 2C1 = 5 x 2 x 2 x 2 x 2 = 80

5-member commitee

5C5 x 2C1 x 2C1 x 2C1 x 2C1 x 2C1 = 1 x 2 x 2 x 2 x 2 x 2 = 32

The required combinations (OR = union) = 80 + 32 = 112

Without those restrictions, there could have been 10C4 + 10C5 possibilities.

Committee of Couples Read More »