Tutorial Exercises

4.12. Tutorial Exercises#

Here we practice running power analysis

4.12.1. Set up Python libraries#

As usual, run the code cell below to import the relevant Python libraries

# Set-up Python libraries - you need to run this but you don't need to change it
import numpy as np
import matplotlib.pyplot as plt
import scipy.stats as stats
import pandas as pd
import seaborn as sns
sns.set_theme(style='white')
import statsmodels.api as sm
import statsmodels.formula.api as smf
import warnings 
warnings.simplefilter('ignore', category=FutureWarning)

4.12.2. Example 1: \(t\)-test#

A researcher plans to carry out a study on the effect of Targetted Memory Reactivation (TMR) on recall. TMR is a method in which memory consolidation is facilitated by cueing. For example, a participant studies a list of vocabulary whilst listening to music; if the same music is then played whilst the participant sleeps, performance on a vocab test the next day is improved (yes, this is is actually true).

To test the effect of TMR, the researchers will use a within-subjects design in which two lists of vocab are learned with two different music tracks. Only one track will be played during sleep. A \(t\)-test will be used to compare scores on next-day vocab tests fot the two lists.

a. Design

What kind of design is this and what type of \(t\)-test will be used?

Your Text Here

b. Recover effect size from literature

The researcher looks at a published paper using a similar task, which reports the results of a \(t\)-test as follows:

There was a significant difference in the number of items recalled between the TMR list and the non-TMR list (\(t_{58}=2.94\), p=0.0023; paired samples \(t\)-test, one tailed)

i) Sample size

We can work out that the sample size is 59 - how?

Your Text Here

ii) \(d\) from \(t\)

Use the equation (see previous pages) to obtain Cohen’s \(d\) from the \(t\)-score

# Your code here

c. Conduct power analysis

Use the built-in power analysis function in statsmodels to work out what sample size we need to get a power of 80%, testing at the \(\alpha=0.05\) level (one tailed)

*Your Code Here*

  Cell In[3], line 1
    *Your Code Here*
          ^
SyntaxError: invalid syntax

You should find that you need 44 participants.

d. Post hoc power analysis

Say the researcher has already run the study with 25 participants (oh dear). What is the power of the study?

*Your Code Here*

power = 0.5854116793095336

You should find the power is 58%. So there is only a 58% chance that we would detect a statistically significant effect of TMR, even if there is indeed an underlying effect.

Example 2: t-test#

A researcher compares the tail length of mice from islands A and B. He reports the findings as follows:

The tails of mice on island B (n=22) were significantly shorter than those on island A (n=30); \(t_{50}=-2.03, p=0.047\), independent samples t-test, two-tailed.

a. Recover effect size

What is the effect size, Cohen’s \(d\)?

*Your Code Here*

0.5698043257969965

You should have found an effect size of 0.57.

b. Was the study sufficiently powered?

Run a power analysis to find the power of the study when using \(\alpha=0.05\)

*Your Code Here*

power = 0.5126655950673099

You should find the power is 51%, so the study was rather under powered.

c. What sample size is necessary for a properly powered study?

The researcher decides to run a new, larger study. What sample size is required to achieve 80% power? Assume equal numbers of mice in each sample.

*Your Code Here*

n = 49.29455400764748

You should find that you need 50 mice per sample for 80% power.

Example 3: Correlation#

In a sample of newlywed couples in Iowa in 2004, IQ was found to be correlated between husbands and wives \(r=0.42, n=276\) couples.

A researcher would like to know if this is also true in Oxford in the current time.

a.Effect size

What is the effect size for the original study?

The effect size is simply the correlation coefficient, \(r=0.42\)

b. Convert \(r\) to \(t\)

To run a power analysis on a correlation using statsmodels, we need to convert our correlation to a \(t\)-score. Look up the equation in the notes and implement it in the code block below to obtain \(t\).

*Your Code Here*

7.660663795726892

c. Obtain Cohen’s \(d\)

Convert the \(t\) score to a \(d\) value

*Your Code Here*

0.46111777312592434

d. Power analysis

We can now use the power analysis for a paired \(t\)-test, on our \(d\) value.

Work out the sample size required for 80% power.

You will have to decide if the test is one-tailed or two-tailed (do we expect only positive correlations, or positive and negative?)

*Your Code Here*

n = 30.49086300941353

We expect a positive correlation - higher IQ husbands have higher IQ wives

Contents