ATTN Analytics I Read Nate Silver "The Signal and the Noise"!

[Philo Sofee]

Sorry man, I couldn't find where you mentioned it and said it was a good read. Indeed! So dang good I couldn't put it down! How absolutely fascinating! I just bought it yesterday, ayiyi. Man this young guy is incredible isn't he. I think my favorite part of the book was definitely his interpretation of Tetlock's Fox and the Hedgehog. That was fantastic!

However, I must confess, perhaps the most informative part of the book for me was his analysis of Climate Change. Just wow, there is more complexity than I ever imagined. Yeah his chapter on baseball was also exquisitely interesting, so MUCH going into just finding the right player! I just had no idea.... wow.

He has a lot of really cool one liners I can see I will be using as I post. This guy writes very good. THANK YOU for recommending this. Overall I am broadened into an understanding that there really is, in reality, a vastly wider swath of use for Bayes than I imagined. And, as he says, this has to widen if we are to get better with our predictions, especially on terrorism, nuclear war, and weather... all fun stuff! My good heck I found a LOT of parallels with Mormon apologetics in his work on weather, the attack on Pearl Harbor, and the Fox and Hedgehog. His specifics can easily be generalized into helping out apologists who desperately need to begin being Bayesians so their thinking straightens out.
His wrap up on why Bayes genuinely can help us out was fantastic as well. Generalized, but quite powerful.
What was it in the book that caught your eye?

[Analytics]

Hi Philo,

I'm so glad you like it! I always get a bit nervous when I recommend or gift a book--reading a book is a significant investment in time, and you are putting yourself out there for criticism.

What really caught my imagination in the book is the way that Nate redefines statistical problems. I realize this depends upon the field and the context of the problem, but too often, people will construct a model, and even if their is a stochastic element to it, they don't fully acknowledge all of the assumptions that go into it, and then don't state the results in a completely honest way. For example, a traditional political pollster will conduct a survey and will calculate the margin of error in the poll. She'll then say something like, we predict Clinton is going to win, because in our poll Clinton is winning by four points, and the poll has a margin of error of only 3 points.

In that example, the "margin of error" might represent a 95% confidence interval or something like that, and if the pollster wanted to get wonky, he could say, "Our model says Clinton is going to win with a p-value of 0.017" (which would mean there was a 1.7% chance that they were really unlikely in their random sample of voters, and Clinton's opponent won the actual election despite the results of the poll).

The problem with that approach is it doesn't explicitly acknowledge the assumptions that went into it, much less deal with them appropriately. The big one, of course, is that the people who respond to any given poll are not a random sampler of voters.

The quote from the book I have etched in my brain more than any other is this one:

Nevertheless, this book encourages readers to think carefully about the signal and the noise and to seek out forecasts that couch their predictions in percentage or probabilistic terms. They are a more honest representation of the limits of our predictive abilities. When a prediction about a complex phenomenon is expressed with a great deal of confidence, it may be a sign that the forecaster has not thought through the problem carefully, has overfit his statistical model, or is more interested in making a name for himself than in getting at the truth. (Page 405)

[Philo Sofee]

A most excellent quote, which applies to me! I must be more careful as I am already recognizing assumptions I am putting into some arguments I make. I am so very fallible it's mildly terrifying to me as I explore into this new kind of area of thinking I am experiencing and thoroughly enjoying in life.

I also found his comment on p. 451 very nicely put - "Information becomes knowledge only when it's placed in context. Without it we have no way to differentiate the signal from the noise, and our search for the truth might be swamped with false positives. What isn't acceptable under Bayes theorem is to pretend that you don't have any prior beliefs. You should work to reduce your biases, but to say you have none is a signal that you have many. To state your beliefs up front - to say 'Here's where I am coming from' - is a way to operate in good faith and to recognize that you perceive reality through a subjective filter." (p. 451) And on the previous page leading up to this grand finale is the sentence "The virtue in thinking probabilistically is that you will force yourself to stop and smell the data - slow down, and consider the imperfections in your thinking."

It is precisely this which has led to my change of mind based, as always upon the upgrading due to evidence factors I had, as an apologist, truly just ignored, since they didn't support the data I wanted them to.

Another thing I really liked in his book was all his charts and graphs! They really helped me visualize a lot better how the information can be presented in other ways that is extra helpful. The charts and graphs in his chapter "Less, and less, and less Wrong," were very helpful to get a visual anchor on Bayes for me.

[Analytics]

Imagine a disciple of Nate Silver who knows nothing about Mormonism. Two missionaries knock on his door, and he is inclined to give their message serious consideration. What should he do?

The first thing he needs to do is set the prior probability for their message being true. The need to do this can seem troubling, but if the logical coherence of Bayes’s formula has clicked in your brain, you realize it must be done in order to be logical about it. The Bayesian investigator might say his prior probability of it being true is one out of a thousand, because that is his guess for the worldwide percentage of Mormons. Or he might set it to two out of three, because two out of three people in his living room believe it. Whatever he sets it to, he realizes that if the Church being true has implications for how the world should be, the evidence he gathers will swamp this a priori.

As the investigation proceeds, the missionaries will quickly seek to help him feel and recognize the spirit. They will say something like, “You feel at peace? Relaxed? Calm? You are sitting in a comfortable chair thinking about things that distract you from the cares of the world? Your heart rate is below 80 bpm? That’s the spirit telling you our message is true!” The Bayesian might respond to this with, “Okay, I’ll contrast how I feel with how you say I’m supposed to feel if it is true. But then I also have to consider how I ought to feel if it is false; how would I expect to feel if my feelings are best explained by sociology, physiology, and psychology?” As he considers the evidence, be it the alleged spiritual evidence and everything else, he needs to honestly and skillfully evaluate it under both paradigms: would we expect this bit of evidence if it were true? Would we expect this bit of evidence if it were false?”

On the second part with regards to the alleged spiritual evidence, the Bayesian would need to understand some things about cognitive science and psychology. Are his emotional reactions consistent with what psychologists and sociologists would expect when somebody goes through the process of socializing themselves into a Mormon mindset and community?

Contrasting how a good Bayesian would investigate the Church with the way missionaries actually guide investigations shows a fundamental fallacy in how missionaries condition their targets to think.

[Analytics]

What isn't acceptable under Bayes theorem is to pretend that you don't have any prior beliefs. You should work to reduce your biases, but to say you have none is a signal that you have many.

[Analytics]

I think the underlying strength of Bayesian thinking is illustrated by the infamous analysis by Dale and Dale of how the Book of Mormon compares to Michael Coe’s The Maya. Even when the apologists select the data, setting up the problem as an explicitly Bayesian one allowed the reader to systematically evaluate the quality of their arguments, as was illustrated in the comments. The reason the critics were able to destroy the Dales’ poor logic was because their Bayesian formulas made the details of their analysis explicit.

[Philo Sofee]

Imagine a disciple of Nate Silver who knows nothing about Mormonism. Two missionaries knock on his door, and he is inclined to give their message serious consideration. What should he do?

The first thing he needs to do is set the prior probability for their message being true. The need to do this can seem troubling, but if the logical coherence of Bayes’s formula has clicked in your brain, you realize it must be done in order to be logical about it. The Bayesian investigator might say his prior probability of it being true is one out of a thousand, because that is his guess for the worldwide percentage of Mormons. Or he might set it to two out of three, because two out of three people in his living room believe it. Whatever he sets it to, he realizes that if the Church being true has implications for how the world should be, the evidence he gathers will swamp this a priori.

As the investigation proceeds, the missionaries will quickly seek to help him feel and recognize the spirit. They will say something like, “You feel at peace? Relaxed? Calm? You are sitting in a comfortable chair thinking about things that distract you from the cares of the world? Your heart rate is below 80 bpm? That’s the spirit telling you our message is true!” The Bayesian might respond to this with, “Okay, I’ll contrast how I feel with how you say I’m supposed to feel if it is true. But then I also have to consider how I ought to feel if it is false; how would I expect to feel if my feelings are best explained by sociology, physiology, and psychology?” As he considers the evidence, be it the alleged spiritual evidence and everything else, he needs to honestly and skillfully evaluate it under both paradigms: would we expect this bit of evidence if it were true? Would we expect this bit of evidence if it were false?”

On the second part with regards to the alleged spiritual evidence, the Bayesian would need to understand some things about cognitive science and psychology. Are his emotional reactions consistent with what psychologists and sociologists would expect when somebody goes through the process of socializing themselves into a Mormon mindset and community?

Contrasting how a good Bayesian would investigate the Church with the way missionaries actually guide investigations shows a fundamental fallacy in how missionaries condition their targets to think.

This, it appears to me, is a very good illustration man. Thanks for puttin this out there. Yes, it shows there are other ways, better ways than merely believing what you are told as the only option. When presented with a hypothesis, instead of going with believing, have the presence of mind to say to yourself, "All right! Now I have something to test, analyze, explore, and learn with."

And, the missionaries cannot be faulted, they are victims as well. Smith set it up from the very beginning to condition his audiences first. The witnesses to the Book of Mormon is a nigh perfect example of this. He had told them before hand (and this is Dan Vogel's singular most IMPORTANT contribution to this whole fiasco is demonstrating this kind of pre-conditioning of people's mindsets) what would happen. Well, sure enough, after enough prodding, and enough times trying, they go the exact kind of revelation Joseph Smith told them would be available to them! And, no one, but no one was going to argue with the Prophet back then. If he said this is how God works, then by God you better get in line and make sure it's how it works for you also. We find the same phenomena at the Kirtland Temple revelations of angels and Jesus Christ. Joseph Smith FIRST describe the vast outpouring of revelation to expect for all the congregation, then on the big day, HE described HIS vision FIRST, so, of course, the self-fulfilling visions just tumbled out of the clouds into the people's minds. This same event is the one where William McClellan wrote to Orson Pratt saying come on dude, the gig is up! We all know, YOU especially, we got so drunk on the sacramental wine before hand, that visions were a foregone conclusion! It was one of the huge turning points of McClellan being turned off by the entire fiasco.

Once I got that new knowledge, of course, updating my knowledge literally changed how I read the D&C record of it all. Very selective, very limited, and entirely and only the faith promoting interpretation of Joseph Smith with very precious little actual historic background of the reality, were allowed into that scripture! The new evidence entirely changes the background knowledge, of which, we, according to Bayesian thinking now, are never allowed to ignore the McClellan/Pratt correspondence ever again in assessing the truth of the situation. It is part of the probability of calculation from here going forward. Man Bayes being all-inclusive really does improve our ability to spot problems and solve them doesn't it. No wonder Packer was so terrified of historians who wanted to tell the WHOLE truth! What a dork he was. Learning Bayes now shows me how quite little I can justifiably put much trust into anything those ole white dudes pontificate on... ya almost feel sorry for em how truly limited and weak their outlook really actually is on reality. I am so blessed to have the medicine of Bayes to cure me of that mind disease.

[huckelberry]

This certainly sounds like an interesting and valuable book. Philo, your comments help me understand why you are finding Bayes a useful device. You are pointing out that it can clarify thinking for the user. It perhaps sounds less useful to a post reader who reads, Bayes computation shows Book of Mormon is clearly a document of ancient historical reality.

Considering its use in a more problematic and controversial question, like was there a actual real person Jesus, can see the process of evaluating evidence both sides as valuable. Perhaps most valuable for the person assigning values.

[Philo Sofee]

I think the underlying strength of Bayesian thinking is illustrated by the infamous analysis by Dale and Dale of how the Book of Mormon compares to Michael Coe’s The Maya. Even when the apologists select the data, setting up the problem as an explicitly Bayesian one allowed the reader to systematically evaluate the quality of their arguments, as was illustrated in the comments. The reason the critics were able to destroy the Dales’ poor logic was because their Bayesian formulas made the details of their analysis explicit.

This was one major issue I discovered on my own thinking and belief, this darn thing exposes the details of my belief and exposes whether I actually am ok with believing what I thought. It just led me to the idea that hey, I actually do have to do some more thinking on all this man. Bayes does open ourselves to our own biases. That is what happened and happens. We can't anymore do so much ignoring of everything we just don't like or are uncomfortable with. Yes, your point is well established.

[Philo Sofee]

This certainly sounds like an interesting and valuable book. Philo, your comments help me understand why you are finding Bayes a useful device. You are pointing out that it can clarify thinking for the user. It perhaps sounds less useful to a post reader who reads, Bayes computation shows Book of Mormon is clearly a document of ancient historical reality.

Considering its use in a more problematic and controversial question, like was there a actual real person Jesus, can see the process of evaluating evidence both sides as valuable. Perhaps most valuable for the person assigning values.

The interesting thing is he actually just works through why he became such a very good and now, well known, predicter of the future! He doesn't do a boatload of math at all. He is describing WHY his predictions are paid attention to with a lot more care than many others! He exposes to himself his own assumptions as much as possible in so many diverse areas such as weather forecasting, nuclear possible wars, baseball, the stock market, Poker, climate change, television political pundits, etc., and in an enormous amount of cases, so many others are making too many assumptions, and are over confident in their knowledge, and make predictions that don't work. And he gives many reasons why his views have been more accurate (sometimes not by a lot, but its there), and why and how others began consulting him saying more or less, ok, what's your gig? And he says I think like a fox, and a Bayesian, (and he shows how in really decent detail actually). I check my assumptions, I put my biases as best to my ability, absolutely ON HOLD, and be strictly honest with the data, as best I can, with, of course, my fallibility. In the long run and truth, there is no more valid way. (and it has made him filthy rich - good Lord he made $400,000 with online poker when it was legal!) Sure there is some herd mentality, he discusses why and when it is good to go with it, and how and why and when to not do so, and more importantly HOW to determine when either decision is more likely to benefit you.

Well, along I come, snatch up his book, read it with laughter, wonder, awe, some anger, some mirth, and realize, gosh dang it man, This applies to me in any area I live in and work through in my intellectual, spiritual, psychological life also! I get it! Astoundingly, I can see no limit to the applications of this kind of thinking which will save time, angst, and anger, not to mention years when I go ahead and begin straightening out my thinking a little more here and now. A beautiful thing as I see it.