Main Page
Welcome to the explain xkcd wiki!
We have an explanation for all 1478 xkcd comics,
and only 16
(1%) are incomplete. Help us finish them!
Latest comic
PValues 
Title text: If all else fails, use "significant at a p>0.05 level" and hope no one notices. 
Explanation
This explanation may be incomplete or incorrect: Needs work to improve readability for nonstatisticians. 
This comic plays on how the significance of scientific experiments is measured and interpreted. The pvalue is a statistical measure of how well the results of an experiment fit with the results predicted by the hypothesis. Low pvalues occur when the results appear to reject the null hypothesis, whereas the high pvalues suggest no relation between the hypothesis and the real world. The pvalue calculated from the experiment data is used to interpret whether the experiment was significant and supports the hypothesis.
Appropriate experimental design generally requires that the significance threshold (usually 0.05) be set prior the experiment, not allowing expost changes in order to get a better experiment report. A simple change of this threshold (e.g. from 0.05 to 0.1) can change the experiment result with pvalue=0.06 from "barely significant" to "significant".
The highest pvalue at which most studies typically draw significance is p<0.05, which is why all pvalues in the comic below that number are marked at least significant. 0.050 is labeled "Oh crap. Redo calculations," because the pvalue is very close to being considered significant, but isn't. Redoing the calculations may result in a different answer, but it is not guaranteed that it will be lower than 0.050. Values that are higher than 0.050 and lower than .1 are considered to be suggesting significance without actually supporting it, which will likely support additional trials.
Values higher than .1 should be considered not significant at all, however the comic suggests taking a part of the sample (a "subgroup") and analyzing that subgroup without regard to the rest of the sample. For example, in a study trying to prove that people always sneeze when walking by a particular street lamp, someone would record the number of people who pass the lamp and the number of people who sneeze. If the results don't get the desired p<0.1, then pick a subgroup (e.g. OK, not all people sneeze, but look! women sneeze more than men, so let's analyze only women). Of course, this is not accepted scientific procedure as it's very likely to add sampling bias to the result. This is an example of the Multiple comparisons problem.
The title text suggests that, if the results cannot be normally considered significant, to invert p<0.050, making it p>0.050. This is intended to fool casual readers, as the change is only to the inequality sign, which may go unnoticed.
Transcript
This explanation may be incomplete or incorrect: First draft. 
There are two columns in a Ttable labelled "pvalue" and "interpretation". The interpretation column selects various areas of the Pvalue column.
Pvalue  Interpretation 

0.001  Highly significant 
0.01  
0.02  
0.03  
0.04  Significant 
0.049  
0.050  Oh crap. Redo calculations. 
0.051  On the edge of significance 
0.06  
0.07  Highly suggestive, relevant at the p<0.10 level 
0.08  
0.09  
0.099  
≥0.1  Hey, look at this interesting subgroup analysis 
Is this out of date?
.New here?
Last 7 days (Top 10) 


You can read a brief introduction about this wiki at explain xkcd. Feel free to sign up for an account and contribute to the wiki! We need explanations for comics, characters, themes and everything in between. If it is referenced in an xkcd web comic, it should be here.
 If you're new to wikis like this, see how to edit wiki pages. There's also a handy wikicode cheatsheet.
 Discussion about the wiki itself happens at the Community portal.
 You can browse the comics from List of all comics or by navigating the category tree at Category:Comics.
 There are incomplete explanations listed here. Feel free to help out by expanding them!
 We sell advertising space to pay for our server costs. To learn more, go here.
Rules
Don't be a jerk.
There are a lot of comics that don't have set in stone explanations; feel free to put multiple interpretations in the wiki page for each comic.
If you want to talk about a specific comic, use its discussion page.
Please only submit material directly related to (and helping everyone better understand) xkcd... and of course only submit material that can legally be posted (and freely edited). Offtopic or other inappropriate content is subject to removal or modification at admin discretion, and users who repeatedly post such content will be blocked.
If you need assistance from an admin, post a message to the Admin requests board.