4.12. Tutorial Exercises#
Here we practice running power analysis
4.12.1. Set up Python libraries#
As usual, run the code cell below to import the relevant Python libraries
# Set-up Python libraries - you need to run this but you don't need to change it
import numpy as np
import matplotlib.pyplot as plt
import scipy.stats as stats
import pandas
import seaborn as sns
import statsmodels.api as sm
import statsmodels.formula.api as smf
4.12.2. Example 1: \(t\)-test#
A researcher plans to carry out a study on the effect of Targetted Memory Reactivation (TMR) on recall. TMR is a method in which memory consolidation is facilitated by cueing. For example, a participant studies a list of vocabulary whilst listening to music; if the same music is then played whilst the participant sleeps, performance on a vocab test the next day is improved (yes, this is is actually true).
To test the effect of TMR, the researchers will use a within-subjects design in which two lists of vocab are learned with two different music tracks. Only one track will be played during sleep. A \(t\)-test will be used to compare scores on next-day vocab tests fot the two lists.
a. Design
What kind of design is this and what type of \(t\)-test will be used?
Your answer here
b. Recover effect size from literature
The researcher looks at a published paper using a similar task, which reports the results of a \(t\)-test as follows:
There was a significant difference in the number of items recalled between the TMR list and the non-TMR list (\(t_{58}=2.94\), p=0.0023; paired samples \(t\)-test, one tailed)
i) Sample size
We can work out that the sample size is 59 - how?
Your answer here
ii) \(d\) from \(t\)
Use the equation (see previous pages) to obtain Cohen’s \(d\) from the \(t\)-score
# Your code here
You should find d=0.38
c. Conduct power analysis
Use the built-in power analysis function in statsmodels to work out what sample size we need to get a power of 80%, testing at the \(\alpha=0.05\) level (one tailed)
# Your code here
You should find that you need 44 participants.
d. Post hoc power analysis
Say the researcher has already run the study with 20 participants (oh dear). What is the power of the study?
# Your code here
You should find the power is 58%. So there is only a 58% chance that we would detect a statistically significant effect of TMR, even if there is indeed an underlying effect.
Example 2: t-test#
A researcher compares the tail length of mice from islands A and B. He reports the findings as follows:
The tails of mice on island B (n=22) were significantly shorter than those on island A (n=30); \(t_{50}=-2.03, p=0.047\), independent samples t-test, two-tailed.
a. Recover effect size
What is the effect size, Cohen’s \(d\)?
# Your code here
You should have found an effect size of 0.57.
b. Was the study sufficiently powered?
Run a power analysis to find the power of the study when using \(\alpha=0.05\)
# Your code here
You should find the power is 51%, so the study was rather under powered.
c. What sample size is necessary for a properly powered study?
The researcher decides to run a new, larger study. What sample size is required to achieve 80% power? Assume equal numbers of mice in each sample.
# Your code here
You should find that you need 50 mice per sample for 80% power.
Example 3: Correlation#
In a sample of newlywed couples in Iowa in 2004, IQ was found to be correlated between husbands and wives \(r=0.42, n=276\) couples.
A researcher would like to know if this is also true in Oxford in the current time.
a.Effect size
What is the effect size for the original study?
Your text here
b. Convert \(r\) to \(t\)
To run a power analysis on a correlation using statsmodels, we need to convert our correlation to a \(t\)-score. Look up the equation in the notes and implement it in the code block below to obtain \(t\).
# Your code here
You should find \(t\)=7.66
c. Obtain Cohen’s \(d\)
Convert the \(t\) score to a \(d\) value
# Your code here
d. Power analysis
We can now use the power analysis for a paired \(t\)-test, on our \(d\) value.
Work out the sample size required for 80% power.
You will have to decide if the test is one-tailed or two-tailed (do we expect only positive correlations, or positive and negative?)
# Your code here