1234231587678 at 23:50, 8 March 2024

2024-03-08T23:50:24Z

Natg19: /* Transcript */

2022-08-09T02:01:47Z

‎Transcript

Jacky720: rv

2022-05-04T23:54:54Z

Show changes

Ex Kay Cee Dee at 23:54, 4 May 2022

2022-05-04T23:54:51Z

Show changes

Jacky720: rv

2022-05-04T23:34:53Z

Show changes

Ex Kay Cee Dee at 23:34, 4 May 2022

2022-05-04T23:34:49Z

Show changes

Jacky720: rv

2022-05-04T21:48:50Z

Show changes

Ex Kay Cee Dee at 21:48, 4 May 2022

2022-05-04T21:48:43Z

Show changes

Qoiuoiuoiu: Undo revision 243939 by Ex Kay Cee Dee (talk)

2022-05-04T20:59:59Z

Undo revision 243939 by Ex Kay Cee Dee (talk)

Show changes

Ex Kay Cee Dee at 20:40, 4 May 2022

2022-05-04T20:40:11Z

Show changes

@@ Line 24: / Line 24: @@
 Values between 0.07 and 0.099 continue the trend of using qualifying language, calling the results "suggestive" or "relevant". This category also illustrates the 'technique' of resorting to adjusting the significance threshold. Appropriate {{w|Design of experiments|experimental design}} requires that the significance threshold be set prior to the experiment, not allowing changes afterward in order to "get a better experiment report", as this would again insert bias into the result. A simple change of the threshold (e.g. from 0.05 to 0.1) can change an experiment's result from "not significant" to "significant". Although the statement "significant at the ''p''<0.10 level" is technically true, it would be highly frowned upon to use in an actual report.
-Values higher than 0.1 are usually considered not significant at all, however the comic suggests taking a part of the sample (a ''subgroup'') and analyzing that subgroup without regard to the rest of the sample. Choosing to analyze a subgroup ''in advance for scientifically plausible reasons'' is good practice. For example, a drug to prevent heart attacks is likely to benefit men more than women, since men are more likely to have heart attacks. Choosing to focus on a subgroup after conducting an experiment may also be valid if there is a credible scientific justification - sometimes researchers learn something new from experiments. However, the danger is that it is usually possible to find and pick an arbitrary subgroup that happens to have a better ''p''-value simply due to chance. A researcher reporting results for subgroups that have little scientific basis (the pill only benefits people with black hair, or only people who took it on a Wednesday, etc.) would clearly be "cheating." Even when the subgroup has a plausible scientific justification, skeptics will rightly be suspicious that the researcher might have considered numerous possible subgroups (men, older people, fat people, sedentary people, diabetes suffers, etc.) and only reported the subgroups for which there are statistically significant results. This is an example of the {{w|multiple comparisons problem}}, which is also the topic of comic [[882]].
+Values higher than 0.1 are usually considered not significant at all, however the comic suggests taking a part of the sample (a ''subgroup'') and analyzing that subgroup without regard to the rest of the sample. Choosing to analyze a subgroup ''in advance for scientifically plausible reasons'' is good practice. For example, a drug to prevent heart attacks is likely to benefit men more than women, since men are more likely to have heart attacks. Choosing to focus on a subgroup after conducting an experiment may also be valid if there is a credible scientific justification - sometimes researchers learn something new from experiments. However, the danger is that it is usually possible to find and pick an arbitrary subgroup that happens to have a better ''p''-value simply due to chance. A researcher reporting results for subgroups that have little scientific basis (the pill only benefits people with black hair, or only people who took it on a Wednesday, etc.) would clearly be "cheating." Even when the subgroup has a plausible scientific justification, skeptics will rightly be suspicious that the researcher might have considered numerous possible subgroups (men, older people, fat people, sedentary people, diabetes suffers, etc.) and only reported the subgroups for which there are statistically significant results. This is an example of the {{w|multiple comparisons problem}}, which is also the topic of [[882: Significant]].
 If the results cannot be normally considered significant, the title text suggests as a last resort to invert p<0.050, making it p>0.050. This leaves the statement mathematically true, but may fool casual readers, as the single-character change may go unnoticed or be dismissed as a typographical error ("no one would claim their results aren't significant, they must mean p<0.050"). Of course, the statement on its face is useless, as it is equivalent to stating that the results are "not significant".

← Older revision		Revision as of 02:01, 9 August 2022
Line 63:		Line 63:
	[[Category:Charts]]		[[Category:Charts]]
	[[Category:Statistics]]		[[Category:Statistics]]
		+	[[Category:Scientific research]]

1478: P-Values - Revision history

1234231587678 at 23:50, 8 March 2024

Natg19: /* Transcript */

Jacky720: rv

Ex Kay Cee Dee at 23:54, 4 May 2022

Jacky720: rv

Ex Kay Cee Dee at 23:34, 4 May 2022

Jacky720: rv

Ex Kay Cee Dee at 21:48, 4 May 2022

Qoiuoiuoiu: Undo revision 243939 by Ex Kay Cee Dee (talk)

Ex Kay Cee Dee at 20:40, 4 May 2022