Explain xkcd: It's 'cause you're dumb.
Welcome to the explain xkcd wiki!
We have an explanation for all 1478 xkcd comics,
and only 0
(0%) are incomplete. Help us finish them!
Go to this comic explanation
|| This explanation may be incomplete or incorrect: Needs work to improve readability for non-statisticians.|
This comic plays on how the significance of scientific experiments is measured and interpreted. The p-value is a statistical measure of how well the results of an experiment fit with the results predicted by the hypothesis. In lay terms, p is the probability that random chance can explain the results, without the experimental prediction. When low p-values occur, the results appear to reject the null hypothesis, whereas high p-values indicate that the data can not be used to support the hypothesis. High p-values do not signify counter-evidence, but only that more results are needed.
Appropriate experimental design generally requires that the significance threshold (usually 0.05) be set prior the experiment, not allowing ex-post changes in order to get a better experiment report. A simple change of this threshold (e.g. from 0.05 to 0.1) can change the experiment result with p-value=0.06 from "not significant" to "significant".
The highest p-value at which most studies typically draw significance is p<0.05, which is why all p-values in the comic below that number are marked at least significant.
It is usually the case that the person carrying out the test has a vested interest in the results, typically because it is their own hypothesis under test. A result which shows no significance can feel like a major blow, and this may lead to desperate attempts to 'encourage' the data to show the desired outcome.
The chart has a p-value of 0.050 labeled "Oh crap. Redo calculations" because the p-value is very close to being considered significant, but isn't. The desperate researcher might be able to redo the calculations in order to nudge the result under 0.050. This could be achieved validly if an error is found in the calculations or data set, or falsely by erasing certain unwelcome data points or by using creative mathematical adjustments.
Values between 0.051 and 0.06 are labelled as being 'On the edge of significance'. The use of this kind of language to qualify the significance is regularly seen in reports, although it is a contested topic. The debate centres on whether p-values slightly larger than the significance level should be noted as nearly-significant, or flatly classed as not-significant. The logic of having an absolute cut-off point for significance is also questioned.
The values between 0.07 and 0.099 continue the trend of using qualifying language, calling the results 'suggestive'. This category also uses another method that the desperate researcher may resort to, namely adjusting the significance threshold. Although it is true that these results are significant at p<0.10, changing the threshold in order to classify the result as significant is highly frowned upon.
Values higher than 0.1 should be considered not significant at all, however the comic suggests taking a part of the sample (a "subgroup") and analyzing that subgroup without regard to the rest of the sample. For example, in a study trying to prove that people always sneeze when walking by a particular street lamp, someone would record the number of people who pass the lamp and the number of people who sneeze. If the results don't get the desired p<0.1, then pick a subgroup (e.g. OK, not all people sneeze, but look! women sneeze more than men, so let's analyze only women). Of course, this is not accepted scientific procedure as it's very likely to add sampling bias to the result. This is an example of the multiple comparisons problem, which is also the topic of comic 882.
If the results cannot be normally considered significant, the title text suggests inverting p<0.050, making it p>0.050. This may fool casual readers, as the change is only to the inequality sign, which may go unnoticed or be dismissed as a typographical error ("no-one would claim their results aren't significant, they must mean p<0.050").
- [A two columns T-table where the interpretation column selects various areas of the first column using square brackets.]
|| Highly significant
|| Oh crap. Redo calculations.
|| On the edge of significance
|| Highly suggestive, relevant at the p<0.10 level
|| Hey, look at this interesting subgroup analysis
Is this out of date? .
Lots of people
contribute to make this wiki a success. Many of the recent contributors, listed above, have just joined. You can do it too! Create your account here
You can read a brief introduction about this wiki at explain xkcd. Feel free to sign up for an account and contribute to the wiki! We need explanations for comics, characters, themes, memes and everything in between. If it is referenced in an xkcd web comic, it should be here.
- List of all comics contains a table of most recent xkcd comics and links to the rest, and the corresponding explanations. There are incomplete explanations listed here. Feel free to help out by expanding them!
- If you see that a new comic hasn't been explained yet, you can create it: Here's how.
- We sell advertising space to pay for our server costs. To learn more, go here.
Don't be a jerk. There are a lot of comics that don't have set in stone explanations; feel free to put multiple interpretations in the wiki page for each comic.
If you want to talk about a specific comic, use its discussion page.
Please only submit material directly related to —and helping everyone better understand— xkcd... and of course only submit material that can legally be posted (and freely edited). Off-topic or other inappropriate content is subject to removal or modification at admin discretion, and users who repeatedly post such content will be blocked.
If you need assistance from an admin, post a message to the Admin requests board.