Data & Statistics

Sleeping Beauty problem

Here is the question: Sleeping Beauty has agreed to be part of an experiment. On Sunday night, she will be put to sleep, and then a fair coin will be flipped. If the coin comes up heads, she will be awakened on Monday and put back to sleep. If the coin comes up tail, she will be awakened on Monday and put back to sleep. She will again be awakened on Tuesday and put back to sleep. Once asleep, she won’t have any memory that she was ever awakened. The experiment ends on Wednesday, and she can go home.

When she wakes up during the experiment, a question is asked: what is the probability that the coin came up heads? So what is her answer? Note that she has complete knowledge (of the rules) of the experiment.

The halfer position

Since she knows the coin is fair, we can always say the probability of heads is (1/2) by disregarding the prospect of having asked this question under three possible scenarios. It is an unconditional probability.

The thirder position

In the absence of any information on the day she woke up, she could imagine three possibilities with equal chances – Monday heads, Monday tails and Tuesday tails. Out of these, one corresponds to heads. So the probability is (1/3).

So there are two possible answers to one question. Well, that is not true. The two questions are different. Asking for the probability of heads can give 1/2 as the answer. The second question was about the probability, given she is awake. To put it differently, what is the chance she is awake because the coin landed heads?

Watch veritasium’s video on this topic.

Sleeping Beauty problem Read More »

The Easterlin Paradox

The Easterlin Paradox, named after economist Richard Easterlin, addresses the relationship between happiness and income. First, the expected part: there is a positive relationship between the real income of people and their happiness – both among countries and within countries. In other words, the richer are happier.

But when it comes to changes in income or living standard as a function of time, happiness tends to remain flat. For example, the real (inflation-adjusted) per capita GDP of the United States was $25,083 in 1972, which became more than doubled ($56,000) by 2018. Yet, the Percentage of people who responded “very happy” was 30% in 1972 and 32% in 2918, and “pretty happy” moved from 53% to 57% in the General Social Survey. Also, the proportion of people who felt their financial situation changed for better or worse remained the same.

The contradiction in the relationship at a point in time vs as a function of time is the essence of this paradox. A possible explanation lies in the argument that the happiness of one arises from comparison with the other. So, even when the whole society moves up richer, if the individual sees no differential growth of their wealth compared to the neighbour, she could feel no increase in happiness. Another explanation is how fast people get used to wealth, and marginal values no longer give the same thrill – some form of diminishing marginal utility.

References

The General Social Survey: GSS
Real gross domestic product per capita: FRED
Easterlin paradox: Wiki

The Easterlin Paradox Read More »

The behavioural immune system

It is a term introduced by the psychological scientist Mark Schaller, describing mechanisms devised by animals, including humans, to counter microbes that cause infection. A simple example is the repulsion towards rotten food.

The behaviour immune system may be considered complementary to the body’s immunological defence. The latter consumes energy and is reactive; the pathogens first enter, and then the body produces compounds (e.g. antibodies) to counter. But a repulsive smell or taste prevents some from consuming it in the first place.

References

Mark Schaller, Phil. Trans. R. Soc. B (2011) 366, 3418–3426

Behavioural immune system: Wiki

The behavioural immune system Read More »

Attacker-Defender Game

We have seen the rock-paper-scissors game in one of the previous posts. Let’s see an application of this in real life. Imagine there are several roads leading to the city in a trouble-hit area, and the police (or military) want to check for potential weapons carried by the criminals.

In such situations, the authorities have a limited scope to implement a deterministic strategy to block one road and check. The perpetrator will soon figure out the idea and prepare plans to avoid the threat. So, what works for the police is to confuse the attackers by choosing checkpoints randomly. In game theory language, this is called a mixed strategy, where the player has multiple options with positive probability. This is contrasted with pure strategy, in which the player has only one action with a positive probability.

Mixed Nash: Game Theory Online

Attacker-Defender Game Read More »

Illusory Correlation Bias

The illusory correlation refers to the error by the human mind that finds correlations between two events that are not correlated. These are sweeping generalisations by the brain that wants to conclude an event quickly (and efficiently). While we all are victims of this bias – fear of strangers, patterns of momentum in share prices, trust in lucky charms – it is not fun to know that psycho-diagnosticians, too, make these errors.

DAP and Rorschach

The pioneering works of Chapman and Chapman analysed popular psychodiagnostic tests such as Draw-a-Person (DAP) and Rorschach in the 60s. In one such study, the researchers presented drawings and descriptions of random symptom statements to a batch of practising clinicians and naive observers. You may know that psycho-diagnosticians have standard checklists correlating the behavioural characteristics of patients with their sketches. And it was no surprise that the observers fell for the stereotypes, just as the clinicians, correlations between random drawings of broad shoulders with manliness, large heads with intelligence etc.

Chapman, L. J.; Chapman, J. P., 1967, Genesis of popular but erroneous psychodiagnostic observations, J Abnorm Psychol, 72(3), 193

Chapman, L. J.; Chapman, J. P., 1969. Illusory correlation as an obstacle to the use of valid psychodiagnostic signs, J Abnorm Psychol, 74(3), 271

Illusory Correlation Bias Read More »

Bayes’ Theorem in a Flu Season

At the peak of flu season, one in a hundred gets the flu. But half of the infected show no symptoms. People with allergies or colds can also show Flu symptoms; one in twenty people who don’t have flu can show flu-like symptoms. So the question is: if a person shows signs of the flu, what is the probability that she has the flu?

We will use this example to illustrate problem-solving through Bayes’ rule. So what is the ask here? You have to tell the chance that a person has flu, given she is showing symptoms, S. Or in shorthand, P(F|S). But what do all we know about flu?

  1. If a random person is picked from the street, there is a one in a hundred chance that he has flu. In other words, P(F) = 1/100 = 0.01. It also means that 99 out of 100 random people in the street have no flu, P(nF) = 0.99.
  2. Only half of the people who have flu show any symptoms. The probability of expressing signs given the person has flu = 0.5. In shorthand, it is P(S|F).
  3. 1 in 20 people who don’t have flu can show flu-like symptoms or P(S|nF) = 0.2

Use all the above information and plug it into Bayes’ equation.

P(F|S) = P(S|F) x P(F) /[P(S|F) x P(F) + P(S|nF) x P(nF)] = 0.5 x 0.01 /[0.5 x 0.01 + 0.2 x 0.99] = 0.0246 = 2.5%

Bayes’ Theorem in a Flu Season Read More »

Flow Model of Happiness

What makes us happy is a question that people have been raising over the years. Psychologist Mihaly Csikszentmihalyi has an answer. He describes the state of feeling happy by the term, flow.

He examined hundreds of people about what made them happy and found out the conditions for flow.

  1. Intense focus on a task. A person focuses on one activity that she forgets about everything else.
  2. Freedom from all self-scrutiny. The job is not imposed upon, but it’s own choice. No fear, no doubt.
  3. Get immediate feedback. An actor gets it when her opposite person reacts; a climber knows how far she must climb before the summit.
  4. Optimally challenging: neither over-challenging nor under-challenging.

Flow Model of Happiness Read More »

Spoilt for Choices

Can you imagine a world without choices? The link between intrinsic happiness and the availability of choice is almost a given. But there is also the feeling of confusion and indecisiveness caused by having too many options. The well-known study by Sheena Iyengar addresses this topic in her research and uncovers some interesting facts about life.

In one of the studies, the research team set up two booths outside a grocery shop, one at a time. A collection of six flavours of jams was on display in the first instance, and there were twenty-four in the second. The researchers then recorded two aspects – the initial motivation to taste and subsequent purchase – that the customer showed.

jams, marmalades, farmers market-997593.jpg

The first observation confirmed the existing belief. The number of options displayed affected the customer’s attention. The 24-flavour booth attracted 60% (145) of the total (242) customers who passed by. Whereas to the limited booth, it was 40% (104 out of 260). Then something strange happened: only 4 of the 145 (3%) purchased a product from the 24-booth, and 31 of the 104 (30%) purchased from the limited booth.

The paradox of choice

Greater availability to choose from indeed makes us happier. At the same time, the greater the choice, the higher our expectations, and we struggle to make decisions.

Iyengar and Lepper, “When Choice is Demotivating: Can One Desire Too Much of a Good Thing?, Journal of Personality and Social Psychology”, 2000, 79(6), 995-1006

Spoilt for Choices Read More »

The Planning Fallacy

The term planning fallacy is closely related to something we have seen earlier – the Dunning–Kruger effect. If the latter is all about overestimating own abilities, the former is about underestimating the amount of time it takes to complete tasks. The story of the Sydney Opera House is a classic example. Located at the foreshores of Sydney Harbour, the iconic building is a venue for performing arts.

sydney, opera, house-4078776.jpg

As per the initial planning in 1957, the cost was about 7 mln dollars and was expected to finish by 1963. Eventually, the building was completed 10 years later than the plan (1973), after spending about 102 mln dollars!

History is full of stories of such mega projects running late and over budget. Yet planners keep missing deadlines and have optimistic estimates of costs. Theoretical analyses propose people who focus on singular information (personal experience) and concentrate on how to complete their task are bound to make errors compared to those who study distributional information (others’ experience).

Three features are typical among people who commit the planning fallacy: 1) too much focus on forward prediction, 2) discounting of issues with personal performances and 3) failure to incorporate relevant knowledge or ‘base rates’ from other projects. Another two personal characteristics associated with poor planners are conditionality and anchoring. In the former case, the assessor focuses on normal operating conditions and ignores the possibility of adverse situations, e.g. war, depression or societal interventions. Anchoring happens when the person sticks to the best guess ignoring the extreme probabilities.

References

Sydney Opera House: Wiki
Buehler, R.; Griffin, D.; Ross, M., Exploring the “Planning Fallacy”: Why People Underestimate Their Task Completion Times, Journal of Personality and Social Psychology, 1994, 67(3), 366-381.
Kahneman, D.; Tversky, A., Intuitive Prediction: Biases and Corrective Procedures, TIMS Studies in Management Science, 1979, 12, 313-327.

The Planning Fallacy Read More »

Probability of Green Balls – Final Episode

Five types of ball-picking problems are introduced here. Welcome to another episode of arrangements (and more confusion). A bag contains 4 green balls and 4 red balls. You like to pick four balls at random, in which:

One green ball

Probability to choose one green AND 3 reds = [4C1 x 4C3] / [8C4]. Remember, the multiplication in the numerator happened because of AND.

At least one green ball

At least one green = P([1 green AND rest red] OR [2 green AND rest red] OR [3 green AND rest red] OR [4 green]). Applying multiplication for AND and summation for OR,

Probability to choose at least one green = [4C1 x 4C3 + 4C2 x 4C2 + 4C3 x 4C1 + 4C4] / [8C4]

At most one green ball

At most one green = P([0 green AND rest red] OR [1 green AND rest red].
Probability to choose at most one green = [4C4 + 4C1 x 4C3] / [8C4]

No green ball

This one is easy; just pick red balls, 4C4/8C4

One green or two greens

What is the probability to pick one green AND three reds or two greens AND two reds?

[4C1 x 4C3 + 4C2 x 4C2] / [8C4]

Probability of Green Balls – Final Episode Read More »