4.11. Concepts Review#
Here we review some conceptual points from the lecture.
Please try to answer each question yourself before clicking to reveal the answer
You can discuss these points with your tutor at the computer-based tutorial session (these sessions are for discussing concepts as well as for developing Python skills)
- What are type 1 and type 2 errors
Click to reveal answer
A type 1 error occurs when the null hypothesis is true, but is rejected (false positive) A type 2 error occurs when the alternative hypothesis is true, but we fail to reject the null (false negative)
- How is statistical power defined?
Click to reveal answer
It is the probability of detecting a statistically significant effect, if there is a real effect in the data
- How does the probability of Type 1 error depend on sample size?
Click to reveal answer
In general it doesn’t as the critical value for statistical tests is chosen to fix the level of Type 1 errors at (say) 5%
- How does the probability of Type 2 error depend on sample size?
Click to reveal answer
Type 2 errors are more likely for small sample sizes - in small samples a common phenomenon is that there is an effect in the expected direction (eg a positive correlation), but the effect is not statistically significant
- The power of a test depends on three variables, what are they?
Click to reveal answer
Effect size, sample size, alpha value
- How is effect size defined for a correlation, and for a t-test
Click to reveal answer
The effect size for a correlation is the correlation coefficient r
The effect size for a t-test is Cohen’s d (see the workbooks for formula)
- How does effect size depend on sample size?
Click to reveal answer
It doesn’t. For example a correlation of r=0.49 represents a certain (large) effect size. You can have a correlation of 0.49 with 5 datapoints or with 500, the effect size is the same. However if you get a p-value for the two correlations, the one based on 500 datapoints will come out much more significnat, as statistical significance depends on both effect size and sample size
- Power of 80% is generally considered desirable. How does the sample
size required to achieve 80% power depend on the effect size?
Click to reveal answer
For a smaller effect size you need a larger sample size to obtain 80% power.
For example, if the effect size is huge - say r=0.9 for a correlation or d=3 for a t-test, you will get 80% power with even a small sample (fewer than 10 individuals)
In contrast if the effect size is tiny - say r=0.1 or d=0.1, you would need a very large sample (hundreds or thousands of individuals) to achieve 80% power.