Editing 1132: Frequentists vs. Bayesians
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 7: | Line 7: | ||
==Explanation== | ==Explanation== | ||
− | + | {{incomplete|The core subject matter, as well as the interpretation of the last panel, is open to debate.}} | |
− | + | This is a comic about probability theory. | |
− | |||
− | + | During the night, a person cannot directly observe the sun to see if it has exploded, but can tell indirectly by a variety of means, many of them simple and practical. For example, the person could use a telephone to call a person in a place where it is day, read posts on Twitter, or look at the moon (the moon does not make its own light, and appears bright only because of reflectd sunlight). In the comic, a person relies on the fact that neutrinos can pass through the earth, so a neutrino detector would detect neutrinos from the sun at all times, day and night. Although this is a theoretically possible method of detecting solar explosions, it is exceedingly impractical, especially using a detector of the size depected in the comic. (If current neutrino detection technology was scaled down to the size shown, and the sun did not become a supernova, then the average rate of neutrino detection would be less than one per week, so waiting for dawn would be faster than waiting for a neutrino to be detected. If the sun did become a supernova, then the entire earth, including the detector, would be instantly destroyed, so the mere fact that the detector survives to give any response is sufficient to conclude, with 100% certainty, that the sun has not become a supernova.) In addition, the detector is stated to give false results ("lie") 1/36th of the time. Assuming this detector is otherwise reliable, when the detector reports a solar explosion, there, there are two possibilities: either (a) the sun has exploded (which is extremely unlikely), but has not become a supernova), and the detector is telling the truth, or (b) the sun ''hasn't'' exploded and the detector is lying (which occurs 1/36th of the time). | |
− | Since the | + | The Frequentist considers what he knows about the detector. Since the detector rolls two standard dice and only lies if they both land on 6, there is only a 1/36 chance that the detector is lying. He references the concept of {{w|P-value|p}}<0.05, which is a scientific research standard where a result is presumed to provide strong evidence against a "null hypothesis" if there is less than a 5% chance that the result occurs given that the null hypothesis is true. (For instance, if you test a new medicine and find that it appears to help your test subjects, and you find that, statistically speaking, the chance that the test subjects improved from the placebo effect alone is less than 5%, you would consider this strong evidence that the medicine is really working.) He notes that the P-value in this case is less than 0.05, and thus the standard threshold has been met. Simply put, the Frequentist notes that it is unlikely for the detector to lie, and therefore the sun has probably exploded. |
− | + | The Bayesian uses a more comprehensive approach. Based on what he knows about the detector, it is unlikely that the detector is lying. But based on what he knows about the ''sun'' (and possibly the relative improbability that the a solar explosion would be detected by the neutrino detector before the expected time of dawn, without being destroyed by a supernova), it is ''extremely'' unlikely that the sun has suddenly exploded. (Modern astronomy tells us that the sun will retain its current condition for at least 5 billion years, aside from minor variations in its output.) The unlikeliness of the detector lying is greatly outweighed by the unlikeliness of the sun exploding. (In Bayesian reasoning, in this context, the knowledge about the probability of the sun exploding is called a {{w|Prior probability|"prior"}}.) Therefore, he concludes that the sun has ''not'' exploded and the detector is lying. (This line of reasoning is not made explicit in the comic, but it is typical of how an ordinary Bayesian would approach the situation.) | |
− | The line, "Bet you $50 it hasn't", | + | The Bayesian's line, "Bet you $50 it hasn't", could be taken as a simple expression of confidence, based on the reasoning above. It could also be taken to mean that the Bayesian has had a further thought: If the sun ''has'' exploded, civilization will quickly collapse and money will become worthless. Thus, even if he loses the bet, he really loses nothing at all. This again references the idea that Bayesians tend to consider things in context, whereas Frequentists have a narrow focus. (It's also a tongue-in-cheek reference to the absurdity of the premise.) |
− | + | The title and the last two frames suggest that "frequentist" interpretation of statistics is somehow wrong, which has prompted debate. Many believe that the Bayesian and the frequentist interpretations of probability theory are not mutually exclusive and neither is wrong. One argument states that the Frequentist in the comic is actually misusing P-values, in a way that violates standard frequentist practice. P-values are usually used only for numerical values that are known to fall along a specific distribution — in this case, it is used to determine the significance of a discrete event, which is wrong. Others believe that the use of prior knowledge by The Bayesian enables him to reach his conclusion. For more views on this issue, see the discussion box. | |
− | The | + | The labels on the bottom two panels were applied as an after-thought, according to Munroe's post [http://web.archive.org/web/20130117080920/http://andrewgelman.com/2012/11/16808/#comment-109366 here]; he states his intention was "to illustrate a case where naïve application of that significance test can give a result that's obviously nonsense." |
− | + | [[File:Nate Silver Tweet.png|.@JoeNBC: If you think it's a toss-up, let's bet. If Obama wins, you donate $1,000 to the American Red Cross. If Romney wins, I do. Deal?|right]] | |
− | |||
− | + | Arguably, this is another comic about the accuracy of presidential election predictions that used Bayesian statistical models, such as Nate Silver's ''538'' and Professor Sam Wang's ''PEC''. Thomas Bayes studied conditional probability — the likelihood that one event is true when given information about some other related event. From {{w|Bayes Theorem|Wikipedia}}: "Bayesian interpretation expresses how a subjective degree of belief should rationally change to account for evidence". The Bayesian's bet may refer to a well-publicized bet that Nate Silver tried to make with Joe Scarborough regarding the outcome of the election (see tweet on the right). | |
− | + | The title text refers to a classic series of logic puzzles known as {{w|Knights and Knaves#Question 3|Knights and Knaves}}, where there are two guards in front of two exit doors, one of which is real and the other leads to death. One guard is a liar and the other tells the truth. The visitor doesn't know which is which, and is allowed to ask one question to one guard. The solution is to ask either guard what the other one would say is the real exit, then choose the opposite. Two such guards were featured in the 1986 Jim Henson movie ''[[246|Labyrinth]]'', which is referenced in the text. | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
==Transcript== | ==Transcript== | ||
− | : | + | :Did the sun just explode? (It's night, so we're not sure) |
− | |||
− | |||
− | :[Two | + | :[Two statisticians stand alongside an adorable little computer that is suspiciously similar to K-9 that speaks in Westminster typeface.] |
:Frequentist Statistician: This neutrino detector measures whether the sun has gone nova. | :Frequentist Statistician: This neutrino detector measures whether the sun has gone nova. | ||
:Bayesian Statistician: Then, it rolls two dice. If they both come up as six, it lies to us. Otherwise, it tells the truth. | :Bayesian Statistician: Then, it rolls two dice. If they both come up as six, it lies to us. Otherwise, it tells the truth. | ||
− | :Frequentist Statistician: Let's try. | + | :Frequentist Statistician: Let's try. [to the detector] Detector! Has the sun gone nova? |
− | : | + | :Detector: ''roll'' YES. |
− | |||
− | |||
:Frequentist Statistician: | :Frequentist Statistician: | ||
:Frequentist Statistician: The probability of this result happening by chance is 1/36=0.027. Since p<0.05, I conclude that the sun has exploded. | :Frequentist Statistician: The probability of this result happening by chance is 1/36=0.027. Since p<0.05, I conclude that the sun has exploded. | ||
Line 61: | Line 43: | ||
:Bayesian Statistician: | :Bayesian Statistician: | ||
:Bayesian Statistician: Bet you $50 it hasn't. | :Bayesian Statistician: Bet you $50 it hasn't. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
{{comic discussion}} | {{comic discussion}} | ||
− | |||
[[Category:Comics featuring Cueball]] | [[Category:Comics featuring Cueball]] | ||
− | [[Category: | + | [[Category:Math]] |
[[Category:Statistics]] | [[Category:Statistics]] | ||
[[Category:Physics]] | [[Category:Physics]] | ||
− |