Friday, February 12, 2010

Randomness

I had a 'podcast day' - a day where I'm sick of listening to music while I work and just want to listen to some interesting talk radio. Unfortunately, my cube is in a dead zone and I only get the radio stations with a really powerful signal, so I've gotten into some really good podcasts - one of which is Radio Lab. Today I listened to an episode from October 2009 called Numbers, followed by one called Stochasticity. I was completely fascinated by both.

It's worth a listen, but the Numbers episode talked about  how childhood development as it pertains to an understanding of numbers, some interesting mathematicians, and Benford's law.  I'd never heard of Benford's law before, but since it's tax time, it seems especially relevant.

The story explained on the Radio Lab episode, which seems to have been sourced directly from this link, is that in 1938 Dr. Benford noticed that a book of logarithm tables was much more heavily worn in the pages with numbers stating with the digit "1" than the other pages.  From this, he began to do some research and discovered that the first digits of a large series of random numbers is naturally weighted in a logritmic scale, with the number 1 occurring more frequently than any other number.  This has amazing applications, including being able to detect fraud on tax returns - if the numbers on your tax return don't follow Benford's law, it's a trigger the IRS could use to investigate more or even perform an audit.

The main take-away from the Stochasticity episode is that people are not very good at understanding randomness - we are people of order.  One of the examples they used was a test where they had two groups of people - one who flipped coins 100 times and recorded the results, and the other who just pretended to flip a coin 100 times and recorded the faked heads/tails sequence.  The fakers were obvious because when the were trying to fake randomness, they wouldn't record 7+ heads or tails in a row.  It just doesn't seem random enough, but the team that really flipped the coin would invariably have those long sequences.

Another example they used is the concept of an athlete being on-fire - a basketball player who gets on a roll and hits 3 in a row starts to get the ball passed to them more often and starts to take riskier shots and invariably winds up hitting a lower percentage of shots than their normal average.  The concept of someone getting on-fire has the exact opposite effect than we intuitively think it would.  We universally misunderstand randomness because we were created as ordered beings in a universe of both order and chaos.