Models and Heuristics
"Irrationality"
here’s one of Kahneman’s first brain twisters:
The mean I.Q. of the population of eighth-graders in a city is known to be 100. You have selected a random sample of 50 children for a study of educational achievement. The first child tested has an I.Q. of 150. What do you expect the mean I.Q. to be for the whole sample?
An I.Q. of 150 is quite rare: It should occur randomly only once out of every 2,330 people. So in this case you might well wonder whether the sample is really “random” or just how confidently it is “known” that the mean is 100.
After all, the United States military severely screwed up the scoring of their I.Q.-like AFQT enlistment test from 1976 to 1980. Senator Sam Nunn kept asking the Pentagon why sergeants were complaining to him that the military was suddenly letting in some real dumb-asses.
The brass, however, scoffed at Nunn’s lowly informants. Obviously, the sergeants were irrationally biased. What could drill instructors possibly know about psychometrics?
But after several years of denial, the Pentagon suddenly announced that their psychologists had accidentally inflated the test’s scoring.
Yet, according to Kahneman, it is irrational for you to worry about real-world concerns like these. He has stipulated that the sample is random and the mean is 100, so that’s all you need to know. [ https://www.takimag.com/article/michael_lewis_hot_hand_steve_sailer/ Steve Sailer]
Heuristics
Pedro Domingos @pmddomingos "It's easy to forget that every cognitive bias is the flip side of a heuristic that works." 6:00 PM · Feb 21, 2021
We need to remember that a heuristic isn't that different from a model. Both of them are highly useful but must be used with care, especially when the regime has just changed.
That's where academics have their advantage--just after regime change. We generally have worse heuristics-- too cumbersome-- but we are better at making new ones, and we actually understand the models. In fact, non-scholars use models at their peril.
Nonscholars use fancy models at their peril--- if you don't understand what you're doing, a simple heuristic is better, as more robust. "The Formula That Killed Wall Street" in WIRED.
Jānis Pipars @k_zars
- "
And hard to recognize that it works the same way with AI, when there is a shift in environment. Deep learning results in de facto heuristic, and neural network has de facto cognitive bias due to same reasons humans do."
