When are Bayesian methods preferable to Frequentist?

It might not seem so at first, but Naïve Bayes is closely related to the perceptron algorithm. The perceptron adds weights and Naïve Bayes multiplies probabilities, but if you take a logarithm, the latter reduces to the former. Both can be seen as generalizations of simple If… then… rules, where each antecedent can count more or less toward the con

A Bayesian statistician, on the other hand, would say, “Wait a minute. We also need to take into account our prior knowledge about the coin.” Did it come from the neighborhood grocery or a shady gambler?

And we’re Bayesian at a deeper level, too. Our brains, our perception, seem to work by predicting the world – prior probabilities – and updating those predictions with information from our senses: new data. Our conscious experience of the world can be best described as our priors. I predict, therefore I am.

This is perhaps the most important role of Bayes’s rule in statistics: we can estimate the conditional probability directly in one direction, for which our judgment is more reliable, and use mathematics to derive the conditional probability in the other direction, for which our judgment is rather hazy.