I recently read an interesting book based on the following ideas:
What is surprising is that this is not an Internet or web analytics book. It is an Economics–or what they call Freakonomics–book. Many times we can learn a lot about web analytics from other fields.
Listen to Steven Levitt speak about crack
The really difficult thing about web analytics is point no. 5. Most people know that this is a challenge but unfortunately they don’t know that the most dangerous problem is how to measure.
Measuring the wrong way is much too common.
Steve Levitt and Stephen Dubner fall into this same trap.
After analyzing what matters in parenting they find out that having been adopted matters. Studies show that a child’s IQ is much more influenced by the biological parents than the adoptive parents (page 171). Now if we left it at that we would think that the adopting parents don’t have much influence on the adoptive child.
website-testing-–-are-website-owners-the-only-ones-who-dont-want-to-test/Luckily, Levitt decided to dig deeper. Maybe because he didn’t like the results. This happens a lot in web analytics—we don’t like the results so we dig deeper until we find what is really going on. This is ok. However you should also do this when we like the results even though it serves our agenda (see point 4 above). You do want to find out the truth, don’t you?
Getting back to the adoptive baby, although the did poorly in school, another study showed that by the time they became adults they “…veered sharply from the destiny that IQ alone might have predicted. Compared to similar children who were not put up for adoption, the adoptees were fare more likely to attend college to have a well-paid job, and to wait until they were out of their teens before getting married. (page 176) ”
So the adoptive parents did matter after all. It is good that Levitt decide to dig deeper. Or maybe his decision of what to measure was wrong (see point 5 above). Maybe instead of measuring success in school he should have been measuring college attendance, jobs and marriage.
It seems like economists make the same mistakes we web analytics people do.
Read the book to get inspired about web analytics.