882: Significant - Revision history

Lettherebedarklight: /* Transcript */ add Category:Comics with a Spanish translation

2025-12-15T08:57:20Z

‎Transcript: add Category:Comics with a Spanish translation

92.23.2.228: /* Explanation */ Mdashes with more obvious 'not-a-hyphen' spacing.

2025-07-21T10:32:25Z

‎Explanation: Mdashes with more obvious 'not-a-hyphen' spacing.

134.60.67.135: /* Explanation */

2025-07-21T07:17:48Z

‎Explanation

Elektrizikekswerk: /* Explanation */ Having such statements in the explanation of a comic about statistical evidence is rather ironic...

2024-06-26T09:57:31Z

‎Explanation: Having such statements in the explanation of a comic about statistical evidence is rather ironic...

Rebekka: Typo

2023-06-22T12:52:52Z

Typo

Rebekka: Added the subjects of multiple testing and false discovery rate to the statistcal part. Changed flow of paragraphs so that the stats theory is one single paragraph separate from the plot discription. Some minor additional changes of wording.

2023-06-22T12:50:39Z

Added the subjects of multiple testing and false discovery rate to the statistcal part. Changed flow of paragraphs so that the stats theory is one single paragraph separate from the plot discription. Some minor additional changes of wording.

108.162.221.42: /* Explanation */ minor grammatical and punctuation corrections

2023-01-20T07:59:02Z

‎Explanation: minor grammatical and punctuation corrections

172.70.131.161: /* Transcript */

2022-11-16T20:07:27Z

‎Transcript

JLZ0kTC5 at 15:17, 21 October 2022

2022-10-21T15:17:20Z

Natg19: /* Transcript */

2022-08-09T01:41:56Z

‎Transcript

← Older revision		Revision as of 08:57, 15 December 2025
Line 105:		Line 105:
	[[Category:Scientific research]]		[[Category:Scientific research]]
	[[Category:Minecraft]]		[[Category:Minecraft]]
		+	[[Category:Comics with a Spanish translation]]

@@ Line 10: / Line 10: @@
 This comic is about {{w|data dredging}} (aka ''p''-hacking), and the misrepresentation of science and statistics in the media. A girl with a black ponytail comes to [[Cueball]] with her claim that {{w|jelly beans}} cause {{w|acne}}, and Cueball then commissions two scientists (a man with goggles and [[Megan]]) to do some research on the link between jelly beans and acne. They find no link, but in the end the real result of this research is bad news reporting!
-First, some basic statistical theory. Let’s imagine you are trying to find out if jelly beans cause acne. To do this you could find a group of people and randomly split them into two groups—one group who you get to eat lots of jelly beans and a second group who are banned from eating jelly beans. After some time you compare whether the group that eat jelly beans have more acne than those who do not. If more people in the group that eat jelly beans have acne, then you might think that jelly beans cause acne. However, there is a problem.
+First, some basic statistical theory. Let’s imagine you are trying to find out if jelly beans cause acne. To do this you could find a group of people and randomly split them into two groups — one group who you get to eat lots of jelly beans and a second group who are banned from eating jelly beans. After some time you compare whether the group that eat jelly beans have more acne than those who do not. If more people in the group that eat jelly beans have acne, then you might think that jelly beans cause acne. However, there is a problem.
 Some people will suffer from acne whether they eat jelly beans or not, and some will never have acne even if they do eat jelly beans. There is an element of chance in how many people prone to acne are in each group. What if, purely by chance, all the group we selected to eat jelly beans would have had acne anyway while those who didn’t eat jelly beans were the lucky sort of people who never get spots? Then, even if jelly beans did not cause acne, we would conclude that jelly beans did cause acne. Of course, it is very unlikely that all the acne prone people end up in one group by chance, especially if we have enough people in each group. However, to give more confidence in the result of this type of experiment, scientists use statistics to see how likely it is that the result they find is purely by chance. This is known as {{w|statistical hypothesis testing}}. Before we start the experiment, we choose a threshold known as the significance level. In the comic the scientists choose a threshold of 5&thinsp;%. If they find that more of the people who ate jelly beans had acne and the chance it was a purely random result is less than 1 in 20, they will say that jelly beans do cause acne. If, however, the chance that their result was purely by random chance is greater than 5&thinsp;%, they will say they have found no evidence of a link. The important point is this: '''there could still be a 1 in 20 chance that this result was purely a statistical fluke.'''
@@ Line 24: / Line 24: @@
 To elaborate on the statistical theory behind this issue:
 If the probability that each trial gives a false positive result is 1 in 20, then by testing 20 different colors it is now likely that at least one jelly bean test will give a false positive. To be precise, the probability of having ''zero'' false positives in 20 tests is 0.95<sup>20</sup> = 35.85&thinsp;% while the probability of having at least 1 false positive in 20 tests is 64.15&thinsp;% (the probability of having ''zero'' false positive in 21 tests (counting the test without color discrimination) is 0.95<sup>21</sup> = 34.06&thinsp;%).
-In scientific fields that perform many simultaneous tests on large amounts of data it is therefore common to adjust for the effect of {{w|Multiple_comparisons_problem|multiple testing}}; typically by controlling the {{w|False_discovery_rate|False Discovery Rate}} which is the number of (expected) false positives compared to all positive results (here, it would be 1/1&nbsp;= 1). For this, you bundle your tests into a single “test of tests” and adjust your single-test p-values such that the chance of your “test of tests” reporting a significant result falls below a certain threshold. Typically, that threshold is 0.05—the same as the conventional p-value for a single test, and it can be interpreted the same way: that only 1 in 20 “tests of tests” would report a result at this level of significance even if the null hypothesis were true.
+In scientific fields that perform many simultaneous tests on large amounts of data it is therefore common to adjust for the effect of {{w|Multiple_comparisons_problem|multiple testing}}; typically by controlling the {{w|False_discovery_rate|False Discovery Rate}} which is the number of (expected) false positives compared to all positive results (here, it would be 1/1&nbsp;= 1). For this, you bundle your tests into a single “test of tests” and adjust your single-test p-values such that the chance of your “test of tests” reporting a significant result falls below a certain threshold. Typically, that threshold is 0.05 — the same as the conventional p-value for a single test, and it can be interpreted the same way: that only 1 in 20 “tests of tests” would report a result at this level of significance even if the null hypothesis were true.
 Applying the {{w|False_discovery_rate#Benjamini–Hochberg_procedure|Benjamini–Hochberg procedure}}, the lowest p-value of a set of 20 tests would need to be smaller than (1/20)&nbsp;× 0.05&nbsp;= 0.0025 to be accepted as significant. Such an adjustment would likely have prevented the situation depicted in the comic.
-This general situation is (sadly) often an issue with more serious matters than jelly beans and acne—at any one time there are many studies about possible links between substances (e.&thinsp;g. red wine) and illness (e.&thinsp;g. cancer). Because only the positive results get reported, this limits the value any single study has—especially if the mechanism linking the two things is not known.
+This general situation is (sadly) often an issue with more serious matters than jelly beans and acne — at any one time there are many studies about possible links between substances (e.&thinsp;g. red wine) and illness (e.&thinsp;g. cancer). Because only the positive results get reported, this limits the value any single study has — especially if the mechanism linking the two things is not known.
 === p-hacking and bad news reporting in real life ===

@@ Line 10: / Line 10: @@
 This comic is about {{w|data dredging}} (aka ''p''-hacking), and the misrepresentation of science and statistics in the media. A girl with a black ponytail comes to [[Cueball]] with her claim that {{w|jelly beans}} cause {{w|acne}}, and Cueball then commissions two scientists (a man with goggles and [[Megan]]) to do some research on the link between jelly beans and acne. They find no link, but in the end the real result of this research is bad news reporting!
-First, some basic statistical theory. Let's imagine you are trying to find out if jelly beans cause acne. To do this you could find a group of people and randomly split them into two groups - one group who you get to eat lots of jelly beans and a second group who are banned from eating jelly beans. After some time you compare whether the group that eat jelly beans have more acne than those who do not. If more people in the group that eat jelly beans have acne, then you might think that jelly beans cause acne. However, there is a problem.
+First, some basic statistical theory. Let’s imagine you are trying to find out if jelly beans cause acne. To do this you could find a group of people and randomly split them into two groups—one group who you get to eat lots of jelly beans and a second group who are banned from eating jelly beans. After some time you compare whether the group that eat jelly beans have more acne than those who do not. If more people in the group that eat jelly beans have acne, then you might think that jelly beans cause acne. However, there is a problem.
-Some people will suffer from acne whether they eat jelly beans or not, and some will never have acne even if they do eat jelly beans. There is an element of chance in how many people prone to acne are in each group. What if, purely by chance, all the group we selected to eat jelly beans would have had acne anyway while those who didn't eat jelly beans were the lucky sort of people who never get spots? Then, even if jelly beans did not cause acne, we would conclude that jelly beans did cause acne. Of course, it is very unlikely that all the acne prone people end up in one group by chance, especially if we have enough people in each group. However, to give more confidence in the result of this type of experiment, scientists use statistics to see how likely it is that the result they find is purely by chance. This is known as {{w|statistical hypothesis testing}}. Before we start the experiment, we choose a threshold known as the significance level. In the comic the scientists choose a threshold of 5%. If they find that more of the people who ate jelly beans had acne and the chance it was a purely random result is less than 1 in 20, they will say that jelly beans do cause acne. If, however, the chance that their result was purely by random chance is greater than 5%, they will say they have found no evidence of a link. The important point is this – '''there could still be a 1 in 20 chance that this result was purely a statistical fluke'''.
+Some people will suffer from acne whether they eat jelly beans or not, and some will never have acne even if they do eat jelly beans. There is an element of chance in how many people prone to acne are in each group. What if, purely by chance, all the group we selected to eat jelly beans would have had acne anyway while those who didn’t eat jelly beans were the lucky sort of people who never get spots? Then, even if jelly beans did not cause acne, we would conclude that jelly beans did cause acne. Of course, it is very unlikely that all the acne prone people end up in one group by chance, especially if we have enough people in each group. However, to give more confidence in the result of this type of experiment, scientists use statistics to see how likely it is that the result they find is purely by chance. This is known as {{w|statistical hypothesis testing}}. Before we start the experiment, we choose a threshold known as the significance level. In the comic the scientists choose a threshold of 5&thinsp;%. If they find that more of the people who ate jelly beans had acne and the chance it was a purely random result is less than 1 in 20, they will say that jelly beans do cause acne. If, however, the chance that their result was purely by random chance is greater than 5&thinsp;%, they will say they have found no evidence of a link. The important point is this: '''there could still be a 1 in 20 chance that this result was purely a statistical fluke.'''
 At first, the scientists do not want to stop playing ''{{w|Minecraft}}'', but they do eventually start. Minecraft was previously referenced in [[861: Wisdom Teeth]].
-The scientists find no link between jelly beans and acne (the probability that the result is by chance is more than 5% i.e. ''p'' > 0.05), but then Megan and Cueball ask them to see if only one color of jelly beans is responsible. They test 20 different colors, each at a significance level of 5%.
+The scientists find no link between jelly beans and acne (the probability that the result is by chance is more than 5&thinsp;%, i.&thinsp;e. ''p'' > 0.05), but then Megan and Cueball ask them to see if only one color of jelly beans is responsible. They test 20 different colors, each at a significance level of 5&thinsp;%.
-This finding leads to a big newspaper headline saying '''Green Jelly Beans Linked To Acne''' where it is said that they have 95 percent confidence with only a 5% chance of a coincidence. Unfortunately, while the p-values reported by the scientists are (presumably) mathematically correct, the wording in the newspaper is misleading and would only apply if green jelly beans were the only ones tested. Common sense should tell you that when you do a whole bunch of tests, it becomes much more likely that you'll get a false positive result.
+This finding leads to a big newspaper headline saying '''Green Jelly Beans Linked To Acne''' where it is said that they have 95 percent confidence with only a 5&thinsp;% chance of a coincidence. Unfortunately, while the p-values reported by the scientists are (presumably) mathematically correct, the wording in the newspaper is misleading and would only apply if green jelly beans were the only ones tested. Common sense should tell you that when you do a whole bunch of tests, it becomes much more likely that you’ll get a false positive result.
 In the title text, we find out that the scientists {{w|Reproducibility|repeated the experiment}} (another key part of the scientific method), but now they no longer find any evidence for the link between acne and green jelly beans. They try to tell the reporter something, maybe that it was probably a coincidence, but the reporters are not interested since that is not news, and refuse to listen. Instead, they make another major headline from the repeat study saying '''Research conflicted''' (which is not accurate, the scientists doubted their results and had their doubts ''confirmed'') and recommend more study on the link (which is what the scientist just did).
 To elaborate on the statistical theory behind this issue:
-If the probability that each trial gives a false positive result is 1 in 20, then by testing 20 different colors it is now likely that at least one jelly bean test will give a false positive. To be precise, the probability of having ''zero'' false positives in 20 tests is 0.95<sup>20</sup> = 35.85% while the probability of having at least 1 false positive in 20 tests is 64.15% (the probability of having ''zero'' false positive in 21 tests (counting the test without color discrimination) is 0.95<sup>21</sup> = 34.06%).
+If the probability that each trial gives a false positive result is 1 in 20, then by testing 20 different colors it is now likely that at least one jelly bean test will give a false positive. To be precise, the probability of having ''zero'' false positives in 20 tests is 0.95<sup>20</sup> = 35.85&thinsp;% while the probability of having at least 1 false positive in 20 tests is 64.15&thinsp;% (the probability of having ''zero'' false positive in 21 tests (counting the test without color discrimination) is 0.95<sup>21</sup> = 34.06&thinsp;%).
-In scientific fields that perform many simultaneous tests on large amounts of data it is therefore common to adjust for the effect of {{w|Multiple_comparisons_problem|multiple testing}}; typically by controlling the {{w|False_discovery_rate|False Discovery Rate}} which is the number of (expected) false positives compared to all positive results (here, it would be 1/1=1). For this, you bundle your tests into a single "test of tests" and adjust your single-test p-values such that the chance of your ''"test of tests"'' reporting a significant result falls below a certain threshold. Typically, that threshold is 0.05 - the same as the conventional p-value for a single test, and it can be interpreted the same way: that only 1 in 20 "tests of tests" would report a result at this level of significance even if the null hypothesis were true.
+In scientific fields that perform many simultaneous tests on large amounts of data it is therefore common to adjust for the effect of {{w|Multiple_comparisons_problem|multiple testing}}; typically by controlling the {{w|False_discovery_rate|False Discovery Rate}} which is the number of (expected) false positives compared to all positive results (here, it would be 1/1&nbsp;= 1). For this, you bundle your tests into a single “test of tests” and adjust your single-test p-values such that the chance of your “test of tests” reporting a significant result falls below a certain threshold. Typically, that threshold is 0.05—the same as the conventional p-value for a single test, and it can be interpreted the same way: that only 1 in 20 “tests of tests” would report a result at this level of significance even if the null hypothesis were true.
-Applying the {{w|False_discovery_rate#Benjamini–Hochberg_procedure|Benjamini–Hochberg procedure}}, the lowest p-value of a set of 20 tests would need to be smaller than (1/20)*0.05 = 0.0025 to be accepted as significant. Such an adjustment would likely have prevented the situation depicted in the comic.
+Applying the {{w|False_discovery_rate#Benjamini–Hochberg_procedure|Benjamini–Hochberg procedure}}, the lowest p-value of a set of 20 tests would need to be smaller than (1/20)&nbsp;× 0.05&nbsp;= 0.0025 to be accepted as significant. Such an adjustment would likely have prevented the situation depicted in the comic.
-This general situation is (sadly) often an issue with more serious matters than jelly beans and acne – at any one time there are many studies about possible links between substances (e.g. red wine) and illness (e.g. cancer). Because only the positive results get reported, this limits the value any single study has - especially if the mechanism linking the two things is not known.
+This general situation is (sadly) often an issue with more serious matters than jelly beans and acne—at any one time there are many studies about possible links between substances (e.&thinsp;g. red wine) and illness (e.&thinsp;g. cancer). Because only the positive results get reported, this limits the value any single study has—especially if the mechanism linking the two things is not known.
 === p-hacking and bad news reporting in real life ===
-In 2015 some journalists demonstrated the same problem: just how gullible other news outlets are with the same sort of flawed "experimental design": [http://www.washingtonpost.com/news/morning-mix/wp/2015/05/28/how-and-why-a-journalist-tricked-news-outlets-into-thinking-chocolate-makes-you-thin/?hpid=z5 How, and why, a journalist tricked news outlets into thinking chocolate makes you thin - The Washington Post]
+In 2015 some journalists demonstrated the same problem: just how gullible other news outlets are with the same sort of flawed “experimental design”: [http://www.washingtonpost.com/news/morning-mix/wp/2015/05/28/how-and-why-a-journalist-tricked-news-outlets-into-thinking-chocolate-makes-you-thin/?hpid=z5 How, and why, a journalist tricked news outlets into thinking chocolate makes you thin - The Washington Post]
 ==Transcript==

@@ Line 14: / Line 14: @@
 Some people will suffer from acne whether they eat jelly beans or not, and some will never have acne even if they do eat jelly beans. There is an element of chance in how many people prone to acne are in each group. What if, purely by chance, all the group we selected to eat jelly beans would have had acne anyway while those who didn't eat jelly beans were the lucky sort of people who never get spots? Then, even if jelly beans did not cause acne, we would conclude that jelly beans did cause acne. Of course, it is very unlikely that all the acne prone people end up in one group by chance, especially if we have enough people in each group. However, to give more confidence in the result of this type of experiment, scientists use statistics to see how likely it is that the result they find is purely by chance. This is known as {{w|statistical hypothesis testing}}. Before we start the experiment, we choose a threshold known as the significance level. In the comic the scientists choose a threshold of 5%. If they find that more of the people who ate jelly beans had acne and the chance it was a purely random result is less than 1 in 20, they will say that jelly beans do cause acne. If, however, the chance that their result was purely by random chance is greater than 5%, they will say they have found no evidence of a link. The important point is this – '''there could still be a 1 in 20 chance that this result was purely a statistical fluke'''.
-At first, the scientists do not want to stop playing the addictive game ''{{w|Minecraft}}'', but they do eventually start. Minecraft was previously referenced in [[861: Wisdom Teeth]].
+At first, the scientists do not want to stop playing ''{{w|Minecraft}}'', but they do eventually start. Minecraft was previously referenced in [[861: Wisdom Teeth]].
 The scientists find no link between jelly beans and acne (the probability that the result is by chance is more than 5% i.e. ''p'' > 0.05), but then Megan and Cueball ask them to see if only one color of jelly beans is responsible. They test 20 different colors, each at a significance level of 5%.

@@ Line 24: / Line 24: @@
 To elaborate on the statistical theory behind this issue:
 If the probability that each trial gives a false positive result is 1 in 20, then by testing 20 different colors it is now likely that at least one jelly bean test will give a false positive. To be precise, the probability of having ''zero'' false positives in 20 tests is 0.95<sup>20</sup> = 35.85% while the probability of having at least 1 false positive in 20 tests is 64.15% (the probability of having ''zero'' false positive in 21 tests (counting the test without color discrimination) is 0.95<sup>21</sup> = 34.06%).
-In fields dealing that perform many simultaneous tests on large amounts of data it is therefore common to adjust for the effect of {{w|Multiple_comparisons_problem|multiple testing}}; typically by controlling the {{w|False_discovery_rate|False Discovery Rate}} which is the number of (expected) false positives compared to all positive results (here, it would be 1/1=1). For this, you bundle your tests into a single "test of tests" and adjust your single-test p-values such that the chance of your ''"test of tests"'' reporting a significant result falls below a certain threshold. Typically, that threshold is 0.05 - the same as the conventional p-value for a single test, and it can be interpreted the same way: that only 1 in 20 "tests of tests" would report a result at this level of significance even if the null hypothesis were true.
+In scientific fields that perform many simultaneous tests on large amounts of data it is therefore common to adjust for the effect of {{w|Multiple_comparisons_problem|multiple testing}}; typically by controlling the {{w|False_discovery_rate|False Discovery Rate}} which is the number of (expected) false positives compared to all positive results (here, it would be 1/1=1). For this, you bundle your tests into a single "test of tests" and adjust your single-test p-values such that the chance of your ''"test of tests"'' reporting a significant result falls below a certain threshold. Typically, that threshold is 0.05 - the same as the conventional p-value for a single test, and it can be interpreted the same way: that only 1 in 20 "tests of tests" would report a result at this level of significance even if the null hypothesis were true.
 Applying the {{w|False_discovery_rate#Benjamini–Hochberg_procedure|Benjamini–Hochberg procedure}}, the lowest p-value of a set of 20 tests would need to be smaller than (1/20)*0.05 = 0.0025 to be accepted as significant. Such an adjustment would likely have prevented the situation depicted in the comic.

@@ Line 8: / Line 8: @@
 ==Explanation==
-This comic is about {{w|data dredging}} (aka ''p''-hacking), and the misrepresentation of science and statistics in the media. A girl with a black ponytail comes to [[Cueball]] with her claim that {{w|jelly beans}} cause {{w|acne}} and Cueball then commission two scientists (a man with goggles and [[Megan]]) to do some research on the link between jelly beans and acne. They find no link, but in the end the real result of this research is bad news reporting!
+This comic is about {{w|data dredging}} (aka ''p''-hacking), and the misrepresentation of science and statistics in the media. A girl with a black ponytail comes to [[Cueball]] with her claim that {{w|jelly beans}} cause {{w|acne}}, and Cueball then commissions two scientists (a man with goggles and [[Megan]]) to do some research on the link between jelly beans and acne. They find no link, but in the end the real result of this research is bad news reporting!
-First some basic statistical theory. Let's imagine you are trying to find out if jelly beans cause acne. To do this you could find a group of people and randomly split them into two groups - one group who you get to eat lots of jelly beans and a second group who are banned from eating jelly beans. After some time you compare whether the group that eat jelly beans have more acne than those who do not. If more people in the group that eat jelly beans have acne then you might think that jelly beans cause acne. However, there is a problem.
+First, some basic statistical theory. Let's imagine you are trying to find out if jelly beans cause acne. To do this you could find a group of people and randomly split them into two groups - one group who you get to eat lots of jelly beans and a second group who are banned from eating jelly beans. After some time you compare whether the group that eat jelly beans have more acne than those who do not. If more people in the group that eat jelly beans have acne, then you might think that jelly beans cause acne. However, there is a problem.
-Some people will suffer from acne whether they eat jelly beans or not and some will never have acne even if they do eat jelly beans. There is an element of chance in how many people prone to acne are in each group. What if, purely by chance, all the group we selected to eat jelly beans would have had acne anyway while those who didn't eat jelly beans were the lucky sort of people who never get spots? Then, even if jelly beans did not cause acne, we would conclude that jelly beans did cause acne. Of course it is very unlikely that all the acne prone people end up in one group by chance, especially if we have enough people in each group. However, to give more confidence in the result of this type of experiment, scientists use statistics to see how likely it is that the result they find is purely by chance. This is known as {{w|statistical hypothesis testing}}. Before we start the experiment, we choose a threshold known as the significance level. In the comic the scientists choose a threshold of 5%. If they find that more of the people who ate jelly beans had acne and the chance it was a purely random result is less than 1 in 20, they will say that jelly beans do cause acne. If however, the chance that their result was purely by random chance is greater than 5% they will say they have found no evidence of a link. The important point is this – '''there could still be a 1 in 20 chance that this result was purely a statistical fluke'''.
+Some people will suffer from acne whether they eat jelly beans or not, and some will never have acne even if they do eat jelly beans. There is an element of chance in how many people prone to acne are in each group. What if, purely by chance, all the group we selected to eat jelly beans would have had acne anyway while those who didn't eat jelly beans were the lucky sort of people who never get spots? Then, even if jelly beans did not cause acne, we would conclude that jelly beans did cause acne. Of course, it is very unlikely that all the acne prone people end up in one group by chance, especially if we have enough people in each group. However, to give more confidence in the result of this type of experiment, scientists use statistics to see how likely it is that the result they find is purely by chance. This is known as {{w|statistical hypothesis testing}}. Before we start the experiment, we choose a threshold known as the significance level. In the comic the scientists choose a threshold of 5%. If they find that more of the people who ate jelly beans had acne and the chance it was a purely random result is less than 1 in 20, they will say that jelly beans do cause acne. If, however, the chance that their result was purely by random chance is greater than 5%, they will say they have found no evidence of a link. The important point is this – '''there could still be a 1 in 20 chance that this result was purely a statistical fluke'''.
-At first the scientists do not want to stop playing the addictive game ''{{w|Minecraft}}'', but they do eventually start. Minecraft was previously referenced in [[861: Wisdom Teeth]].
+At first, the scientists do not want to stop playing the addictive game ''{{w|Minecraft}}'', but they do eventually start. Minecraft was previously referenced in [[861: Wisdom Teeth]].
-The scientists find no link between jelly beans and acne (the probability that the result is by chance is more than 5% i.e. ''p'' > 0.05) but then Megan and Cueball ask them to see if only one color of jelly beans is responsible. They test 20 different colors each at a significance level of 5%. If the probability that each trial gives a false positive result is 1 in 20, then by testing 20 different colors it is now likely that at least one jelly bean test will give a false positive. To be precise, the probability of having ''no'' false positive in 20 tests is 0.95<sup>20</sup> = 35.85%. Probability of having ''no'' false positive in 21 tests (counting the test without color discrimination) is 0.95<sup>21</sup> = 34.06%.
+The scientists find no link between jelly beans and acne (the probability that the result is by chance is more than 5% i.e. ''p'' > 0.05), but then Megan and Cueball ask them to see if only one color of jelly beans is responsible. They test 20 different colors, each at a significance level of 5%. If the probability that each trial gives a false positive result is 1 in 20, then by testing 20 different colors it is now likely that at least one jelly bean test will give a false positive. To be precise, the probability of having ''no'' false positive in 20 tests is 0.95<sup>20</sup> = 35.85%. Probability of having ''no'' false positive in 21 tests (counting the test without color discrimination) is 0.95<sup>21</sup> = 34.06%.
-This leads to a big newspaper headline saying '''Green Jelly Beans Linked To Acne''' where it is said that they have 95 percent confidence with only a 5% chance of a coincidence.  Unfortunately, although this number has been reported by the scientists' stats package and would be true if green jelly beans were the only ones tested, it is also seriously misleading.  If you roll just one die, one time, you aren't very likely to roll a six... but if you roll it 20 times you are very likely to have at least one six among them.  This means that you cannot just ignore the other 19 experiments that failed.
+This finding leads to a big newspaper headline saying '''Green Jelly Beans Linked To Acne''' where it is said that they have 95 percent confidence with only a 5% chance of a coincidence.  Unfortunately, although this number has been reported by the scientists' stats package and would be true if green jelly beans were the only ones tested, it is also seriously misleading.  If you roll just one die, one time, you aren't very likely to roll a six... but if you roll it 20 times you are very likely to have at least one six among them.  This means that you cannot just ignore the other 19 experiments that failed. There are good methods for handling this problem, notably {{w|Bayesian_inference|Bayesian inference}}, but they can be difficult to use and explain, and complexity does not sell newspapers.
-There are good methods for handling this problem, notably {{w|Bayesian_inference|Bayesian inference}}, but they can be difficult to use and explain, and complexity does not sell newspapers.
+In the title text, we find out that the scientists {{w|Reproducibility|repeated the experiment}} (another key part of the scientific method), but now they no longer find any evidence for the link between acne and green jelly beans. They try to tell the reporter something, maybe that it was probably a coincidence, but the reporters are not interested since that is not news. So they do not listen to what the scientist has to say and instead uses the information they have to make another major headline saying '''Research conflicted''' and recommend more study on the link. But that was just what the scientist already did.
 This is (sadly) often an issue with more serious matters than jelly beans and acne – at any one time there are many studies about possible links between substances (e.g. red wine) and illness (e.g. cancer). Because only the positive results get reported, this limits the value any single study has - especially if the mechanism linking the two things is not known.

@@ Line 32: / Line 32: @@
 :Girl with black ponytail: Jelly beans cause acne!
 :Cueball: Scientists! Investigate!
-:Scientist (off screen): But we're playing Minecraft!
+:Scientist (off panel): But we're playing Minecraft!
-:Scientist (off screen): ...Fine.
+:Scientist (off panel): ...Fine.
 :[Two scientists. The man has safety goggles on, Megan has a sheet of notes.]
@@ Line 44: / Line 44: @@
 :Scientist (off screen): But Miiiinecraft!
-:[20 identical small panels follow, 4 rows 5 columns. The exact same picture as in panel 2 above. The scientist with goggles are stating the results and Megan holds some notes in her hand. The only difference from panel to panel is the color and then in the 14th panel where the result is positive and there is an exclamation from off screen.]
+:[20 identical small panels follow, in 4 rows of 5 columns. The exact same picture as in panel 2 above. The scientist with goggles states the results and Megan holds some notes in her hand. The only difference from panel to panel is the color and then in the 14th panel where the result is positive and there is an exclamation from off-panel.]
 :Scientist with goggles: We found no link between purple jelly beans and acne (p > 0.05).
@@ Line 72: / Line 72: @@
 :Scientist with goggles: We found a link between green jelly beans and acne (p < 0.05).
-:Voice (off screen): ''Whoa!''
+:Voice (off panel): ''Whoa!''
 :Scientist with goggles: We found no link between mauve jelly beans and acne (p > 0.05).
@@ Line 92: / Line 92: @@
 :Only 5% chance of coincidence!
 :Scientists...
 {{comic discussion}}

@@ Line 99: / Line 99: @@
 [[Category:Comics featuring Megan]]
 [[Category:Statistics]]
-[[Category:Research Papers]]
+[[Category:Scientific research]]
 [[Category:Minecraft]]