Item — HN Reader

Gelman makes a distinction between p-hacking and choosing the test based on your data.

p-hacking is confirming your preferred hypothesis - just try again.

the latter let's you write a paper.

(test power is different: if there's really an effect, would you detect it)

1 comments

Oops, I meant to say test size, not test power.

A test size of 0.05 is only really 0.05 if the entire process you used was free from p-hacking. And as a reader of that paper that process includes the file drawer effect. All these things have an impact on distorting the test size away from what it is advertised to be.