2001: Clickbait-Corrected p-Value
Title text: When comparing hypotheses with Bayesian methods, the similar 'clickbayes factor' can account for some harder-to-quantify priors.
| This explanation may be incomplete or incorrect: Language and grammar should be checked by a native English speaker.|
If you can address this issue, please edit the page! Thanks.
Clickbait is the practice of using deceptive or manipulative headlines to entice readers to click on a dubious news story, often with the purpose of generating ad revenue. Randall uses the wide controversy about the health effects of chocolate to humans; many believe it's just minor but others are convinced about greater effects. In fact there are no reliable studies to confirm any effect and no medical authority has approved any positive or negative health claim. But because many people like chocolate those two opposite claims shown here are a guarantee for many clicks on both sides.
Hypothesis testing in statistics is a standard method to determine whether a particular hypothesis is supported by the data. For the topic given in this comic, a researcher might compare data on athletic performance with data on chocolate consumption by those athletes to determine whether the two trend together. By convention, the "null hypothesis" (designated H0) is that there's no correlation (that chocolate isn't correlated with athletic performance, in this case) and the "alternate hypothesis" (H1) is that they are correlated. (If the study consists of feeding chocolate to one of two identical groups and not the other, rather than tracking what they'd be eating anyway, then the alternative hypothesis can be strengthened to be that chocolate *causes* improved performance.) These sets are subjected to statistical tests which return a "test statistic". From that test statistic a "p-value" is calculated. The p-value indicates the probability of observing the obtained results (or any more extreme value), when all assumptions of the test are true (including the null hypothesis).
In layman's terms: The p-value is the probability that the researcher sees results as extreme or more extreme than the observed result given the null hypothesis is true; the p-value is NOT the probability that the null hypothesis is correct. It answers the question: If there is no correlation, how likely was it that I saw a correlation at least this big? Hence, if the p-value is low enough (by convention < 0.05), the null hypothesis is rejected, and we conclude that the alternate hypothesis is supported by the data (NOT that it is "correct" or "true").
In this comic, the p-value is corrected by a factor that takes clickbait into account. This factor has the effect of increasing the p-value if H1 is more clickbaity than H0, and decreases the p-value if H0 is more clickbaity than H1. This suggests that whatever clickers of clickbait believe, the reverse is likely to be true.
Or, another interpretation could be that this factor corrects for a selection bias effect where the p-values for more clickbaity H1s tend to be lower than they should be and p-values for non-clickbaity H0s to be higher than they should be. For example, one explanation could be that for p-values that are on the cusp of significance, researchers may be more incentivized to fudge and adjust the data to get the p-value down if the H1 is highly sensational, since the H1 would make the research more likely to get published and attract attention. (See also FiveThirtyEight's article on p-hacking and this Stack Exchange question about p-hacking in the wild.)
As the statistical results now depend on people's beliefs about the hypothesis, this could appear as far from actual science as one can get. However, in a way, it is more in tune with a quote by Arbuthnot (one of the originators of the use of p-values) attributing variation to active thought rather than chance, "From whence it follows, that it is Art, not Chance, that governs." Randall applying that quote to the thoughts of the masses, brings it in line with "Art".
If this correction could be somehow enforced on the scientific world, it would have the effect of keeping the popular view of scientific results more in line with reality. Often one study will be performed that shows an exciting result, and it will reach the media without any further studies to verify it. If this is a sensational result, people may become excited before learning that the result was in fact false. The clickbait correction aids science by requiring results that would be sensational if published to undergo much more rigorous demonstration. Additionally, there can be a problem in some areas of science where more boring results never undergo the third-party testing necessary to verify their truth or falseness, or perhaps are even never studied in the first place. The clickbait correction factor has the opposite effect on these more boring topics, making it easier to demonstrate effects within them, perhaps in the hope that more will get studied and published.
Technically, the comic's depiction of null and alternative hypotheses is not entirely correct. As the alternative hypothesis (H1) predicts that chocolate will improve performance (i.e., a one-tailed, directional hypothesis), the null hypothesis (H0) should predict that chocolate will do nothing or make performance worse. In other words, the alternative hypothesis should be true if and only if the null hypothesis is false. For example, alternatively, if the H1 were to say that chocolate will change performance (for better or worse; i.e., a two-tailed hypothesis) then H0 should say that chocolate will do nothing.
The title text refers to Bayesian statistics in which the probability is related to a degree of belief in an event and the prior probability, or simply just prior, expresses this belief before an event has happened. An election forecast is a simple example to this. And here it's suggested using the "clickbait factor" click(H1)/click(H0) as an absurd "clickbayes factor" to determine the prior for a prediction.
- [Under a heading that says Clickbait-Corrected p-Value there is a mathematic formula. Below that is the description of the two used variables and what they mean:]
- Clickbait-corrected p-value:
- PCL = Ptraditional ∙ click(H1)/click(H0)
- H0: NULL hypothesis ("Chocolate has no effect on athletic performance")
- H1: Alternative hypothesis ("Chocolate boosts athletic performance")
- click(H): Fraction of test subjects who click on a headline announcing that H is true
add a comment! ⋅ add a topic (use sparingly)! ⋅ refresh comments!