Roman's Data Science: How to monetize your data

In data analysis, survival bias is taking the known into account while neglecting the unknown (which nevertheless exists).

First, we need to know what the goal of the hypothesis is – what quantitative metric will it optimize?

The first thing to be investigated is integration, and this brings us to our first hypothesis: Is the data from the online store transmitted to us correctly? Sixty to seventy percent of problems are typically dealt with at this stage.

nine out of ten hypotheses don’t pan out. But you have no idea that a hypothesis will not produce the desired result until you are well into the testing process. I believe that it is best to kill a hypothesis as early as possible – as soon as the first sign that the idea won’t take off presents itself.

One problem with all of these tests is that they are distribution-specific. For example, the Student’s t-test and the z-test require normally distributed data.

We start out thinking that hypothesis H0 is true until we find evidence to refute it in favour of hypothesis H1.

Bootstrapping is one of the most interesting ways to evaluate metrics in A/B tests. It is one of our favoured methods at Retail Rocket for continuous parameters such as average purchase value, average cost of goods and revenue per visitor (RPV).

Kozyrkov implores us to “always evaluate decision quality based only on what was known at the time the decision was made.”

The parameter in the general population is true, and the sample parameter is an estimate of the true parameter.