The Wisdom of Crowds

I just finished reading Suriweicki's book on how aggregate decision making is better than individual decision making. Riffing off Charles MacKay's "The Madness of Crowds"--which discusses how mob mentality leads to market bubbles, riots, and other forms of badness--Suriweicki goes through all the cases where group behavior, aggregated, do better than individual behavior.

It's a good book and touched on many topics I've been thinking about for the past year or so. He begins by stating that group averages of estimates, such as the number of jelly beans in a jar, are usually much better than most, if not all, individual estimates. He also goes through the markets, democracies, teams, behavioral finance, and the dangers of group think.

I don't feel he went far enough, though, into why groups make better decisions than individuals, and therefore what goes wrong when they start making worse decisions (such as market bubbles).

Often, when people want to take a swing at Markets, they trot out some behavioral finance experiment that shows people are not perfectly rational. This is a poor choice of argument, firstly because people do not need to be perfectly rational for markets to be efficienct, and secondly because it's much more straightforward to simply point out asset price bubbles instead as examples of gross market inefficiency. Whenever the opponent resorts to talking about "animal spirits", you know they are on the defensive.

To me, the behavioral economics helps explain the value of markets, not undermine then. Individual cognition is a weak and biased thing, it's brittleness easily revealed in simple experiments where smart people can be reliably made to make dumb choices. Markets limit, or actively work against humans' natural cognitive limits, and reward those who overcome these limitations best. Humans are just as cognitively flawed in committees, or by themselves, and only the anonymity and abstractness of the market gets them to try and manage these frailties.

For example, individuals over value stuff they have and undervalue stuff they do not have. So, people will not part with a coffee mug they have been given for $5, but will refuse to buy it for $3. Markets abstract away the feeling of ownership (when you buy some Coca-Cola stock, do you really feel a part of the whole Coca-Cola family) so people make better decisions about what the real value is. This disinterested abstract anonymity is one of the many things that the anti-globo mob, but it is instead a real strength.

Creating procedures that debias or rebias our natural cognitive weaknesses is also central to science. Humans, for example, do not seek out disconfirming evidence. When asked to guess the rule behind a number sequence, and allowed to test other number sequences to see if they follow the rule, most people do something like this:

Tester: The sequence is "2, 4, 6". What's the rule?
Subject: Does "6, 8, 10" follow the rule?
Tester: Yes.
Subject: How about "20, 22, 24"?
Tester: Sure.
Subject: OK [Things hard] What about "1000, 1002, 1004"?
Tester: Yes, that follow's the rule too.
Subject: The rule is that the numbers have to go up by 2 each time.
Tester: Wrong.

The subject usually never tries to disconfirm his initial hypothesis. He never tried "1, 2, 3" (which would follow the rule) or "1, 2, 100" (which would also follow the rule) or even "3, 6, 9" (still OK). By now you've probably guessed what the rule is, but please note how the process of actively seeking out disconfirming evidence is contrary to human nature, and then think how the scientific approach, which is to "come up with a hypothesis and then try to disprove it" actively takles that fallibility. Popper and Kuhn were quite right to point out how Science often happens in very unscientific ways, but in light of the above I think they totally miss the point. Nothing can stop humans being human, all you can do is try to build on your strengths and limit your weaknesses.


Firstly, in a market prices are determined by the marginal buyer, who may be rational (or at least, expert), and secondly because it's much easier The best refutation of efficient markets is price bubbles, and Chicago certainly does not have a good answer into why these occur.

The book also did not touch on Arrow's Incompleteness theorm, which reveals the inherent arbitrariness of *any* political process, and which I consider central to a non-stupid discussion of politics in general. But I thought the ending was right on--democracies don't make good decisions, but the decision to have one is very wise indeed. Squabbling and mudslinging are ugly, but they are better than insular decision making.