Imagine this: a peaceful city of a million population wakes up with news of a murder. Its inhabitants, not used to such events, naturally get shocked and think of it as a failure of the establishment. The administration recruits a highly specialised cop from an area of another city notorious for criminals. The place was in such a bad shape that it had 10000 people and 100 criminals, and the officer had a lot of experience catching them.
The cop knows the strength of facial recognition systems as he had a high success rate in catching criminals in his old job. He trusts it was because of the high accuracy of the system, i.e. a 1% false-positive rate and a 100% true positivity (if you are a criminal, you are caught). Is recruiting the top cop a good strategy?
The answer depends on how much of the previous experience the cop was willing to forget and learn the reality of the new city. Look at, mathematically, the problem with his background. We use Bayes’ theorem.
Violent City: prevelance of criminal P(C) = 100/10000 = 0.01, P(+ve|C) = 1, P(+ve|nC) = 0.01, P(nc) = 1 – P(C). Chance of the person is a criminal given the facial recognition is matched, P(C|+ve) = (1 x 0.01) / [1 x 0.01 + 0.01 x 0.99] = 0.5 = 50%; he was right half the time.
Peaceful City: prevelance of criminal P(C) = 100/1000000 = 0.0001, P(+ve|C) = 1, P(+ve|nC) = 0.01, P(nc) = 1 – P(C). P(C|+ve) = (1 x 0.0001) / [1 x 0.0001 + 0.01 x 0.9999] = 0.01 = 1%.
The right level of experience
If the officer uses his experience and starts using a facial recognition system to randomly check people, expect him to catch 99 innocent for every potential criminal. Instead, he can use the tool as supporting evidence for those who are caught for other suspicious activities.
Now replace murder with drinking, facial recognition with a breath analyser. The results will be the same as long as the tool is employed for random checking – a lot of innocents are penalised.
