| Latest revision |
Your text |
| Line 8: |
Line 8: |
| | | | |
| | ==Explanation== | | ==Explanation== |
| − | This comic is another in a [[:Category:COVID-19|series of comics]] related to the {{w|COVID-19 pandemic}}, specifically regarding the [[:Category:COVID-19 vaccine|COVID-19 vaccine]]. It is also another one of [[Randall|Randall's]] [[:Category:Tips|Tips]], this time a statistics tip. The next tip comic after this [[2435: Geothmetic Meandian]] had a stats tip.
| + | {{incomplete|Created by a PLACEBO GROUP. Please mention here why this explanation isn't complete. Do NOT delete this tag too soon.}} |
| | | | |
| − | ===Graph===
| + | This comic is another comic in a [[:Category:COVID-19|series of comics]] related to the {{w|2019–20 coronavirus outbreak|2020 pandemic}} of the {{w|coronavirus}} {{w|SARS-CoV-2}}, which causes {{w|COVID-19}}. |
| − | The main focus of the comic is a graph showing cases of COVID-19 versus time for two groups: one group was vaccinated and the other group was not. Graphs are ways to visualize data, and for real data indicate specific values. This graph seems to be based on [https://www.nejm.org/doi/full/10.1056/nejmoa2034577 the Pfizer vaccine's results]. The higher line ("placebo group") rises in a steep curve. The lower line ("vaccine group") follows the first for a bit but then levels out to a much slower rate of climb. Officially, a scientific assessment of the effectiveness of anything requires rigorous statistical analysis. This is particularly true in medical studies, where impacts of biology can be highly complex and subject to many factors, meaning that careful review of the data is necessary to confirm that an intervention was effective. The joke of this comic is that the intervention presented here is so ''obviously'' effective that it's obvious even to a layman with little understanding of the math. A few days after the vaccine was administered, cases in the vaccinated group essentially flatline, while cases in the placebo group continue to rise as a significant rate. The data is so "good", meaning that numbers for the treatment and control groups diverge so dramatically, that actual analysis becomes almost a formality: a glance at the chart would convince most people that the treatment is effective.
| |
| | | | |
| − | This comic was released one week after the FDA granted an emergency use authorization for the {{w|BNT162b2|Pfizer COVID-19 vaccine}}, and 8 days after results of its Phase 3 clinical trial were published in [https://www.nejm.org/doi/full/10.1056/nejmoa2034577 the ''New England Journal of Medicine'']. The document includes the following [https://www.nejm.org/na101/home/literatum/publisher/mms/journals/content/nejm/2020/nejm_2020.383.issue-27/nejmoa2034577/20210819/images/img_xlarge/nejmoa2034577_f3.jpeg chart]. The charts draw the integral of the incidence data rather than the data itself ("cumulative" rather than "rate"): this results in changes in disease rate towards the left side of the chart, being added into the data on the right side, amplifying their difference. This technique for emphasizing the data is valid: the spread between the lines only continues to increase if the effect continues happening, such that the total spread at the right is proportional to the total effect the vaccine had. The charts do not show any information on other possible variables. Randall has described previously in his webcomics how very clear charts can be made to hide misleading data. The linked graph does not leave the numbers out, and the numbers indicate the vaccine is 91% effective at preventing the disease (and a 95% chance of being between 85 and 95% efficient).
| + | The main focus of the comic is a graph showing cases of COVID-19 versus time for two groups: one group was vaccinated and the other group was not. (It is not clear which COVID-19 vaccine(s) are included in the graph, but it is mostly likely either the Moderna vaccine or the Pfizer vaccine. Graphs are ways to visualize data, and almost always indicate specific values. This graph does not; it simply has two lines. The higher line ("placebo group") rises in a steep curve. The lower line ("vaccine group") follows the first for a bit but then levels out to a much slower rate of climb. The caption eschews statistical analysis in favor of a holistic assessment: the vaccine is clearly working; just look how far apart those lines are. |
| | | | |
| − | The advice here could be seen as the inverse of the "science tip" in [[2311: Confidence Interval]], in which the data was so ''bad'' that its error bars fell outside of the graph and were not shown. Also there's some association with [[1725: Linear Regression]] where the data is not so good that you don't need to perform linear analysis.
| + | This comic was released one day after the [https://www.fda.gov/media/144434/download FDA's Dec 17th briefing document for the Moderna COVID-19 vaccine] was released. The document includes the following chart: [[File:FDA_Modena_Dec17.png]] |
| | | | |
| − | ===Null hypothesis===
| + | The advice here could be seen as the inverse of the "science tip" in [[2311: Confidence Interval]], in which the data was so ''bad'' that its error bars fell outside of the graph and were not shown. |
| − | | |
| − | The null hypothesis, mentioned in the title text, is the hypothesis in a statistical analysis that indicates that the effect investigated by the analysis does not occur, i.e. 'null' as in zero effect. For example, the null hypothesis for this study might be "The vaccine has no effect on whether subjects catch COVID." The null hypothesis was previously the subject of [[892: Null Hypothesis]]. The null hypothesis is rejected when the probability of something like the observed data would be very low were the null hypothesis true. | |
| − | | |
| − | For a simplified example, imagine there are 10 000 people in the vaccinated group, and each has a 5% chance of catching COVID under the null hypothesis; we expect 500 people to catch COVID. If only 490 catch COVID, the null hypothesis remains plausible, but if just 10 do, the odds are (in Python; see {{w|binomial distribution}}) <code>sum([math.comb(10000, i) * 0.05**i * 0.95**(10000-i) for i in range(0,10)])</code> = 1.5 × 10<sup>-204</sup>. In other words, it is wildly improbably that an ineffective vaccine would have produced such excellent results. We therefore conclude that the vaccine is not ineffective, and have rejected the null hypothesis.
| |
| − | | |
| − | Most people however, on seeing the raw results, would have concluded that the vaccine worked and statistics were just a formality. As the title text says, they would have "reject[ed] the null hypothesis based on the 'hot damn, check out this chart' test."
| |
| | | | |
| | ==Transcript== | | ==Transcript== |
| − | :[Shown is a graph with the x-axis labeled "time" and the y-axis labeled "COVID cases." There is a black line on the graph labeled "placebo group", which has a roughly linear slope moving toward the top right corner. There is a red line labeled "vaccine group", which follows the black line for about an eighth of the width of the graph before leveling off at a much slower increase.] | + | {{incomplete transcript|Do NOT delete this tag too soon.}} |
| − | | + | :[Shown is a graph with the x-axis labeled "time" and the y-axis labeled "COVID cases." There is a black line on the graph labeled "placebo group", which has a roughly linear slope moving toward the top right corner. There is a red line labeled "vaccine group", which follows the black line for about an eighth of the width of the graph before leveling off.] |
| − | :Caption beneath the graph: Statistics tip: Always try to get data that's good enough that you don't need to do statistics on it | + | :[Caption beneath the graph]: |
| | + | :Statistics tip: Always try to get data that's good enough that you don't need to do statistics on it |
| | | | |
| | {{comic discussion}} | | {{comic discussion}} |
| Line 34: |
Line 28: |
| | | | |
| | [[Category:COVID-19]] | | [[Category:COVID-19]] |
| − | [[Category:COVID-19 vaccine]]
| |
| | [[Category:Comics with color]] | | [[Category:Comics with color]] |
| | [[Category:Statistics]] | | [[Category:Statistics]] |
| | [[Category:Tips]] | | [[Category:Tips]] |
| − | [[Category:Scientific research]]
| |
