{ "cells": [ { "cell_type": "markdown", "id": "71a8ecd9", "metadata": {}, "source": [ "# Assessing Regression models in Python.\n", "\n", "The process of applying equations for $R^2$, standard errors, and the 95% confidence intervals around the slope are all done for us in Python. In fact, without asking, Python gives us these statistics in the regression output. To have a first look, we will come back to the immigration attitudes data from last week’s tutorial. \n", "\n", "\n", "### Set up Python libraries\n", "\n", "As usual, run the code cell below to import the relevant Python libraries" ] }, { "cell_type": "code", "execution_count": 1, "id": "33a5d3b4", "metadata": {}, "outputs": [], "source": [ "# Set-up Python libraries - you need to run this but you don't need to change it\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import scipy.stats as stats\n", "import pandas \n", "import seaborn as sns\n", "import statsmodels.api as sm\n", "import statsmodels.formula.api as smf" ] }, { "cell_type": "markdown", "id": "e6cb4ebe", "metadata": {}, "source": [ "# ESS data\n", "\n", "Let's work again with the European Social Survey data (attitudes to immigration)\n", "\n", "### Import and view data" ] }, { "cell_type": "code", "execution_count": 2, "id": "3c0954af", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
1votebetterbornuksexageeduc
02Conservative0.00Male75.0NaN
13Conservative7.00Male70.0Upper secondary
24Conservative8.00Female54.0Tertiary
35Conservative0.00Male58.0Upper secondary
46Conservative7.00Male76.0Lower secondary
........................
21992201NaN9.00Female32.0Tertiary
22002202NaN5.00Female69.0Lower secondary
22012203NaN5.00Female34.0Upper secondary
22022204NaN2.00Male23.0Lower secondary
22032205NaN0.00Female54.0Upper secondary
\n", "

2204 rows × 7 columns

\n", "
" ], "text/plain": [ " 1 vote better bornuk sex age educ\n", "0 2 Conservative 0.0 0 Male 75.0 NaN\n", "1 3 Conservative 7.0 0 Male 70.0 Upper secondary\n", "2 4 Conservative 8.0 0 Female 54.0 Tertiary\n", "3 5 Conservative 0.0 0 Male 58.0 Upper secondary\n", "4 6 Conservative 7.0 0 Male 76.0 Lower secondary\n", "... ... ... ... ... ... ... ...\n", "2199 2201 NaN 9.0 0 Female 32.0 Tertiary\n", "2200 2202 NaN 5.0 0 Female 69.0 Lower secondary\n", "2201 2203 NaN 5.0 0 Female 34.0 Upper secondary\n", "2202 2204 NaN 2.0 0 Male 23.0 Lower secondary\n", "2203 2205 NaN 0.0 0 Female 54.0 Upper secondary\n", "\n", "[2204 rows x 7 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# load and view the data\n", "ess = pandas.read_csv('https://raw.githubusercontent.com/jillxoreilly/StatsCourseBook/main/data/immigrationData.csv')\n", "ess" ] }, { "cell_type": "markdown", "id": "c1c999ec", "metadata": {}, "source": [ "We’ll fit a regression model just like we did last week. " ] }, { "cell_type": "code", "execution_count": 3, "id": "ef30f619", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
OLS Regression Results
Dep. Variable: better R-squared: 0.136
Model: OLS Adj. R-squared: 0.134
Method: Least Squares F-statistic: 65.72
Date: Mon, 20 Feb 2023 Prob (F-statistic): 6.87e-64
Time: 11:11:33 Log-Likelihood: -4775.5
No. Observations: 2097 AIC: 9563.
Df Residuals: 2091 BIC: 9597.
Df Model: 5
Covariance Type: nonrobust
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
coef std err t P>|t| [0.025 0.975]
Intercept 5.2565 0.200 26.269 0.000 4.864 5.649
sex[T.Male] 0.0390 0.104 0.375 0.707 -0.165 0.243
educ[T.Tertiary] 1.8464 0.139 13.316 0.000 1.574 2.118
educ[T.Upper secondary] 0.6699 0.127 5.274 0.000 0.421 0.919
age -0.0118 0.003 -4.056 0.000 -0.017 -0.006
bornuk 1.1811 0.155 7.636 0.000 0.878 1.484
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Omnibus: 44.928 Durbin-Watson: 1.905
Prob(Omnibus): 0.000 Jarque-Bera (JB): 46.767
Skew: -0.354 Prob(JB): 6.99e-11
Kurtosis: 2.818 Cond. No. 249.


Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified." ], "text/plain": [ "\n", "\"\"\"\n", " OLS Regression Results \n", "==============================================================================\n", "Dep. Variable: better R-squared: 0.136\n", "Model: OLS Adj. R-squared: 0.134\n", "Method: Least Squares F-statistic: 65.72\n", "Date: Mon, 20 Feb 2023 Prob (F-statistic): 6.87e-64\n", "Time: 11:11:33 Log-Likelihood: -4775.5\n", "No. Observations: 2097 AIC: 9563.\n", "Df Residuals: 2091 BIC: 9597.\n", "Df Model: 5 \n", "Covariance Type: nonrobust \n", "===========================================================================================\n", " coef std err t P>|t| [0.025 0.975]\n", "-------------------------------------------------------------------------------------------\n", "Intercept 5.2565 0.200 26.269 0.000 4.864 5.649\n", "sex[T.Male] 0.0390 0.104 0.375 0.707 -0.165 0.243\n", "educ[T.Tertiary] 1.8464 0.139 13.316 0.000 1.574 2.118\n", "educ[T.Upper secondary] 0.6699 0.127 5.274 0.000 0.421 0.919\n", "age -0.0118 0.003 -4.056 0.000 -0.017 -0.006\n", "bornuk 1.1811 0.155 7.636 0.000 0.878 1.484\n", "==============================================================================\n", "Omnibus: 44.928 Durbin-Watson: 1.905\n", "Prob(Omnibus): 0.000 Jarque-Bera (JB): 46.767\n", "Skew: -0.354 Prob(JB): 6.99e-11\n", "Kurtosis: 2.818 Cond. No. 249.\n", "==============================================================================\n", "\n", "Notes:\n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", "\"\"\"" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Fit a regression model: Y = better, x1 = age, x2, = sex, x3 = education, x4 = bornuk.\n", "\n", "# first we run this line to tell statsmodels where to find the data and the explanatory variables\n", "reg_formula = sm.regression.linear_model.OLS.from_formula(data = ess, formula = 'better ~ age + sex + educ + bornuk')\n", "\n", "# then we run this line to fit the regression (work out the values of intercept and slope)\n", "# the output is a structure which we will call reg_results\n", "reg_results = reg_formula.fit()\n", "\n", "# let's view a summary of the regression results\n", "reg_results.summary() " ] }, { "cell_type": "markdown", "id": "d9cda220", "metadata": {}, "source": [ "### Interpretation\n", "\n", "Have a good look at the contents of the output table above \n", "\n", "* Find the standard error. \n", "* Find the lower and upper confidence interval. \n", "\n", "Answer the quick questions:\n", "\n", "1. What is the slope for age, and what are the confidence intervals around the slope? How would you interpret these confidence intervals in words?\n", "\n", " * $b$ = -.0117951, the 95% confidence intervals are -.0174981 and -.0060921 meaning that we can be 95% confident that the true population value lies between these two points. The lower and upper confidence bounds are below zero, so we can be sure the slope is statistically significant.\n", " \n", " \n", "2. What other information in the table can tell us the slope is statistically significant?\n", "\n", " * The standard error of .002908 can be used to give the $t$-statistic. -.0117951/.002908 = -4.06. As this $t$ value is larger than 1.96, we know the slope is statistically significant. The regression table also provides the $p$-value associated with this $t$-statistic. So, there are several ways to know if the slope is statistically significantly different to zero. \n", " \n", " \n", "3. For all the $x$-variables in this model, find the one(s) that are not statistically significant. \n", " * In this model, the variable for sex is not significant. There is no difference in immigration attitudes between men and women. \n", " \n", " \n", "4. What is the $R^2$ for this model, and how do we interpret it? \n", " * $R^2$ = 0.1358 meaning that 13.6% of the variability in $y$ is explained by these $x$s. \n", " \n", " \n", "5. Remember the model with political party? Do you think the $R^2$ in this model will be higher or lower? Let’s take a look. " ] }, { "cell_type": "code", "execution_count": 4, "id": "4e6d329e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
OLS Regression Results
Dep. Variable: better R-squared: 0.077
Model: OLS Adj. R-squared: 0.073
Method: Least Squares F-statistic: 20.77
Date: Mon, 20 Feb 2023 Prob (F-statistic): 1.71e-23
Time: 11:11:33 Log-Likelihood: -3444.4
No. Observations: 1511 AIC: 6903.
Df Residuals: 1504 BIC: 6940.
Df Model: 6
Covariance Type: nonrobust
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
coef std err t P>|t| [0.025 0.975]
Intercept 6.9235 0.464 14.909 0.000 6.013 7.834
sex[T.Male] 0.0403 0.123 0.329 0.742 -0.200 0.281
vote[T.Conservative] -0.6885 0.589 -1.170 0.242 -1.843 0.466
vote[T.Labour] 1.1944 0.564 2.119 0.034 0.089 2.300
age -0.0145 0.008 -1.823 0.068 -0.030 0.001
age:vote[T.Conservative] -0.0033 0.010 -0.332 0.740 -0.023 0.016
age:vote[T.Labour] -0.0213 0.010 -2.133 0.033 -0.041 -0.002
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Omnibus: 41.378 Durbin-Watson: 1.968
Prob(Omnibus): 0.000 Jarque-Bera (JB): 43.572
Skew: -0.403 Prob(JB): 3.46e-10
Kurtosis: 2.796 Cond. No. 951.


Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified." ], "text/plain": [ "\n", "\"\"\"\n", " OLS Regression Results \n", "==============================================================================\n", "Dep. Variable: better R-squared: 0.077\n", "Model: OLS Adj. R-squared: 0.073\n", "Method: Least Squares F-statistic: 20.77\n", "Date: Mon, 20 Feb 2023 Prob (F-statistic): 1.71e-23\n", "Time: 11:11:33 Log-Likelihood: -3444.4\n", "No. Observations: 1511 AIC: 6903.\n", "Df Residuals: 1504 BIC: 6940.\n", "Df Model: 6 \n", "Covariance Type: nonrobust \n", "============================================================================================\n", " coef std err t P>|t| [0.025 0.975]\n", "--------------------------------------------------------------------------------------------\n", "Intercept 6.9235 0.464 14.909 0.000 6.013 7.834\n", "sex[T.Male] 0.0403 0.123 0.329 0.742 -0.200 0.281\n", "vote[T.Conservative] -0.6885 0.589 -1.170 0.242 -1.843 0.466\n", "vote[T.Labour] 1.1944 0.564 2.119 0.034 0.089 2.300\n", "age -0.0145 0.008 -1.823 0.068 -0.030 0.001\n", "age:vote[T.Conservative] -0.0033 0.010 -0.332 0.740 -0.023 0.016\n", "age:vote[T.Labour] -0.0213 0.010 -2.133 0.033 -0.041 -0.002\n", "==============================================================================\n", "Omnibus: 41.378 Durbin-Watson: 1.968\n", "Prob(Omnibus): 0.000 Jarque-Bera (JB): 43.572\n", "Skew: -0.403 Prob(JB): 3.46e-10\n", "Kurtosis: 2.796 Cond. No. 951.\n", "==============================================================================\n", "\n", "Notes:\n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", "\"\"\"" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Code to run a regression. Y = better, x1 = age, x2, = sex, x3=vote x4 = age*vote.\n", "\n", "# first we run this line to tell statsmodels where to find the data and the explanatory variables\n", "reg_formula = sm.regression.linear_model.OLS.from_formula(data = ess, formula = 'better ~ age + sex + vote + age:vote')\n", "\n", "# then we run this line to fit the regression (work out the values of intercept and slope)\n", "# the output is a structure which we will call reg_results\n", "reg_results = reg_formula.fit()\n", "\n", "# let's view a summary of the regression results\n", "reg_results.summary() " ] }, { "cell_type": "markdown", "id": "de6e309b", "metadata": {}, "source": [ "R-squared for this model is 0.076 meaning that we have now explained around 7.6% of the variation in y, i.e., less than in our first model." ] }, { "cell_type": "markdown", "id": "1fea3a77", "metadata": {}, "source": [ "### RMSE\n", "\n", "We don't have RMSE (root mean squared error) in our summary table, but we can call the MSE from the larger results structure, and get its square root as follows:" ] }, { "cell_type": "code", "execution_count": 10, "id": "638cab2a", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2.370092002233663" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "reg_results.mse_resid**0.5" ] }, { "cell_type": "markdown", "id": "38b5ae76", "metadata": {}, "source": [ "We could compare the RMSE for this regression model to the RMSE we calculated when we only had a single explanatory variable, age (which we did by hand in the section on conditional distributions)\n", "\n", "With only age used as an explanatory variable, RMSE was a bit higher, 2.50\n", "\n", "* What does this suggest about spread of values around the regression line?" ] }, { "cell_type": "markdown", "id": "6c1b9b97", "metadata": {}, "source": [ "## Checking regression assumptions\n", "\n", "In Python, we can also examine the regression assumptions. We can explore:\n", "\n", "* whether the data are suitably linear\n", "* whether there is heteroskedasticity in the residuals\n", "* whether the residuals are normally distributed\n", "\n", "For now, let's just take a look at the last of these three assumptions and plot a histogram of the residuals:" ] }, { "cell_type": "code", "execution_count": 24, "id": "9e2ca194", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAGdCAYAAAD0e7I1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAorElEQVR4nO3df3BU9b3/8dfKjyVggoQlyabml1wwYiwoIJTaS6IYiYpX4SIUYUCR1lukciOjRvQmONb0akV7odoyV34ocGE6t1imMNUAAlq0A0EUMDCgxEVITBchm8C6BHLuH/2yX7chQJKT7NkPz8fMmcn5nLNv33vGqa+e89nzcVmWZQkAAMBQV0S7AQAAgPZE2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGK1ztBtwgsbGRh09elTx8fFyuVzRbgcAAFwCy7JUV1en1NRUXXFF8/dvCDuSjh49qrS0tGi3AQAAWuHw4cO6+uqrmz1O2JEUHx8v6e8XKyEhIcrdAACASxEIBJSWlhb+73hzCDtS+NFVQkICYQcAgBhzsSkoTFAGAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDRWPQeAFvL5fPL7/bbW9Hg8Sk9Pt7UmgL8j7ABAC/h8PmVnX6dg8JStdePiumvfvgoCD9AOCDsA0AJ+v1/B4CkNe6hYCd5MW2oGqir118Xz5Pf7CTtAOyDsAEArJHgzlZh+bbTbAHAJmKAMAACMRtgBAABGI+wAAACjRTXsbN26VWPGjFFqaqpcLpfefvvtiOMul+u820svvRQ+Jzc3t8nxiRMndvA3AQAAThXVsHPy5EkNHDhQCxcuPO/xqqqqiG3x4sVyuVwaN25cxHkzZsyIOO93v/tdR7QPAABiQFR/jVVQUKCCgoJmj6ekpETs//GPf1ReXp6uueaaiPHu3bs3ORcAAECKoTk7X3/9tdatW6fp06c3ObZixQp5PB5df/31mjNnjurq6i5YKxQKKRAIRGwAAMBMMfOenWXLlik+Pl5jx46NGH/ggQeUlZWllJQU7dmzR0VFRfrkk09UVlbWbK3S0lLNmzevvVsGAAAOEDNhZ/HixXrggQfUrVu3iPEZM2aE/87JyVG/fv00ZMgQ7dy5UzfddNN5axUVFamwsDC8HwgElJaW1j6NAwCAqIqJsPP+++9r//79Wr169UXPvemmm9SlSxcdOHCg2bDjdrvldrvtbhMAADhQTMzZeeONNzR48GANHDjwoufu3btXDQ0N8nq9HdAZAABwuqje2amvr9fBgwfD+4cOHdKuXbuUmJgYXgwvEAjo97//vV5++eUmn//888+1YsUK3XnnnfJ4PPrss8/0+OOP68Ybb9QPf/jDDvseAADAuaIadnbs2KG8vLzw/rl5NFOnTtXSpUslSatWrZJlWfrxj3/c5PNdu3bVxo0b9etf/1r19fVKS0vTXXfdpeLiYnXq1KlDvgMAAHC2qIad3NxcWZZ1wXN+8pOf6Cc/+cl5j6WlpWnLli3t0RoAADBETMzZAQAAaC3CDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjRTXsbN26VWPGjFFqaqpcLpfefvvtiOPTpk2Ty+WK2IYPHx5xTigU0qxZs+TxeNSjRw/dc889+uqrrzrwWwAAACeLatg5efKkBg4cqIULFzZ7zujRo1VVVRXe1q9fH3F89uzZWrNmjVatWqUPPvhA9fX1uvvuu3X27Nn2bh8AAMSAztH8hxcUFKigoOCC57jdbqWkpJz3WG1trd544w299dZbGjVqlCRp+fLlSktL04YNG3THHXfY3jMAAIgtjp+zs3nzZiUlJal///6aMWOGampqwsfKy8vV0NCg/Pz88FhqaqpycnK0bdu2ZmuGQiEFAoGIDQAAmMnRYaegoEArVqzQpk2b9PLLL2v79u269dZbFQqFJEnV1dXq2rWrevXqFfG55ORkVVdXN1u3tLRUPXv2DG9paWnt+j0AAED0RPUx1sVMmDAh/HdOTo6GDBmijIwMrVu3TmPHjm32c5ZlyeVyNXu8qKhIhYWF4f1AIEDgAQDAUI6+s/OPvF6vMjIydODAAUlSSkqKTp8+rePHj0ecV1NTo+Tk5GbruN1uJSQkRGwAAMBMMRV2jh07psOHD8vr9UqSBg8erC5duqisrCx8TlVVlfbs2aMRI0ZEq00AAOAgUX2MVV9fr4MHD4b3Dx06pF27dikxMVGJiYkqKSnRuHHj5PV6VVlZqaeffloej0f33XefJKlnz56aPn26Hn/8cfXu3VuJiYmaM2eObrjhhvCvswAAwOUtqmFnx44dysvLC++fm0czdepUvf7669q9e7fefPNNnThxQl6vV3l5eVq9erXi4+PDn3nllVfUuXNn3X///QoGg7rtttu0dOlSderUqcO/DwAAcJ6ohp3c3FxZltXs8XfeeeeiNbp166YFCxZowYIFdrYGAAAMEVNzdgAAAFqKsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGC0ztFuAAAQG3w+n/x+v231PB6P0tPTbasHNIewAwC4KJ/Pp+zs6xQMnrKtZlxcd+3bV0HgQbsj7AAALsrv9ysYPKVhDxUrwZvZ5nqBqkr9dfE8+f1+wg7aHWEHAHDJEryZSky/NtptAC3CBGUAAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBrLRQAAoqaiosLWeqykjvOJatjZunWrXnrpJZWXl6uqqkpr1qzRvffeK0lqaGjQM888o/Xr1+uLL75Qz549NWrUKP3yl79UampquEZubq62bNkSUXfChAlatWpVR34VAEALBGuPSXJp8uTJttZlJXWcT1TDzsmTJzVw4EA9+OCDGjduXMSxU6dOaefOnXr22Wc1cOBAHT9+XLNnz9Y999yjHTt2RJw7Y8YMPffcc+H9uLi4DukfAJzK5/PJ7/fbVs/uOzANp+okWRo06Un1ycq2pSYrqaM5UQ07BQUFKigoOO+xnj17qqysLGJswYIFuvnmm+Xz+SL+Re7evbtSUlLatVcAiBU+n0/Z2dcpGDxle+2G0Glb612ZlM4q6mh3MTVnp7a2Vi6XS1dddVXE+IoVK7R8+XIlJyeroKBAxcXFio+Pb7ZOKBRSKBQK7wcCgfZqGQA6nN/vVzB4SsMeKlaCN9OWmlW7P9SetYt05swZW+oBHSlmws63336rp556SpMmTVJCQkJ4/IEHHlBWVpZSUlK0Z88eFRUV6ZNPPmlyV+i7SktLNW/evI5oGwCiJsGbadtdk0BVpS11gGiIibDT0NCgiRMnqrGxUa+99lrEsRkzZoT/zsnJUb9+/TRkyBDt3LlTN91003nrFRUVqbCwMLwfCASUlpbWPs0DAICocnzYaWho0P33369Dhw5p06ZNEXd1zuemm25Sly5ddODAgWbDjtvtltvtbo92AQCAwzg67JwLOgcOHNB7772n3r17X/Qze/fuVUNDg7xebwd0CAAAnC6qYae+vl4HDx4M7x86dEi7du1SYmKiUlNT9a//+q/auXOn/vSnP+ns2bOqrq6WJCUmJqpr1676/PPPtWLFCt15553yeDz67LPP9Pjjj+vGG2/UD3/4w2h9LQAA4CBRDTs7duxQXl5eeP/cPJqpU6eqpKREa9eulSQNGjQo4nPvvfeecnNz1bVrV23cuFG//vWvVV9fr7S0NN11110qLi5Wp06dOux7AAAA54pq2MnNzZVlWc0ev9AxSUpLS2vy9mQAAIDvYiFQAABgNMIOAAAwmqN/jQXg8mP3mk4SK2EDlzvCDgDHaK81nVgJG7i8EXYAOEZ7rOnEStgACDsAHMfONZ0AgAnKAADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKN1jnYDANARKioqHFUHQMch7AAwWrD2mCSXJk+ebGvdhtBpW+sBaD+EHQBGazhVJ8nSoElPqk9WdpvrVe3+UHvWLtKZM2fa3hyADkHYAXBZuDIpXYnp17a5TqCqsu3NAOhQTFAGAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEZrVdi55pprdOzYsSbjJ06c0DXXXNPmpgAAAOzSqrBTWVmps2fPNhkPhUI6cuRIm5sCAACwS4vWxlq7dm3473feeUc9e/YM7589e1YbN25UZmambc0BAAC0VYvCzr333itJcrlcmjp1asSxLl26KDMzUy+//LJtzQEAALRVix5jNTY2qrGxUenp6aqpqQnvNzY2KhQKaf/+/br77rsvud7WrVs1ZswYpaamyuVy6e233444blmWSkpKlJqaqri4OOXm5mrv3r0R54RCIc2aNUsej0c9evTQPffco6+++qolXwsAABisVXN2Dh06JI/H0+Z/+MmTJzVw4EAtXLjwvMdffPFFzZ8/XwsXLtT27duVkpKi22+/XXV1deFzZs+erTVr1mjVqlX64IMPVF9fr7vvvvu8c4oAAMDlp0WPsb5r48aN2rhxY/gOz3ctXrz4kmoUFBSooKDgvMcsy9Krr76quXPnauzYsZKkZcuWKTk5WStXrtRPf/pT1dbW6o033tBbb72lUaNGSZKWL1+utLQ0bdiwQXfccUdrvx4AADBEq+7szJs3T/n5+dq4caP8fr+OHz8esdnh0KFDqq6uVn5+fnjM7XZr5MiR2rZtmySpvLxcDQ0NEeekpqYqJycnfM75hEIhBQKBiA0AAJipVXd2fvvb32rp0qWaMmWK3f2EVVdXS5KSk5MjxpOTk/Xll1+Gz+natat69erV5Jxznz+f0tJSzZs3z+aOAQCAE7Xqzs7p06c1YsQIu3s5L5fLFbFvWVaTsX90sXOKiopUW1sb3g4fPmxLrwAAwHlaFXYefvhhrVy50u5eIqSkpEhSkzs0NTU14bs9KSkpOn36dJNHZ98953zcbrcSEhIiNgAAYKZWPcb69ttvtWjRIm3YsEHf//731aVLl4jj8+fPb3NjWVlZSklJUVlZmW688UZJf7+jtGXLFv3nf/6nJGnw4MHq0qWLysrKdP/990uSqqqqtGfPHr344ott7gEAAMS+VoWdTz/9VIMGDZIk7dmzJ+LYxR4xfVd9fb0OHjwY3j906JB27dqlxMREpaena/bs2XrhhRfUr18/9evXTy+88IK6d++uSZMmSZJ69uyp6dOn6/HHH1fv3r2VmJioOXPm6IYbbgj/OgsAAFzeWhV23nvvPVv+4Tt27FBeXl54v7CwUJI0depULV26VE888YSCwaB+9rOf6fjx4xo2bJjeffddxcfHhz/zyiuvqHPnzrr//vsVDAZ12223aenSperUqZMtPQIAgNjW6vfs2CE3N1eWZTV73OVyqaSkRCUlJc2e061bNy1YsEALFixohw4BAECsa1XYycvLu+Djqk2bNrW6IQC4XFVUVDiqDmCKVoWdc/N1zmloaNCuXbu0Z8+eJguEAgAuLFh7TJJLkydPtrVuQ+i0rfWAWNWqsPPKK6+cd7ykpET19fVtaggALjcNp+okWRo06Un1ycpuc72q3R9qz9pFOnPmTNubAwxg65ydyZMn6+abb9avfvUrO8sCcDCfzye/329Lrcv98cuVSelKTL+2zXUCVZVtbwYwiK1h58MPP1S3bt3sLAnAwXw+n7Kzr1MweMrWujx+AWCnVoWdc6uQn2NZlqqqqrRjxw49++yztjQGwPn8fr+CwVMa9lCxEryZba7H4xcA7aFVYadnz54R+1dccYWuvfZaPffccxErkAO4PCR4M3n8AsCxWhV2lixZYncfAAAA7aJNc3bKy8tVUVEhl8ulAQMGhNewAgAAcIpWhZ2amhpNnDhRmzdv1lVXXSXLslRbW6u8vDytWrVKffr0sbtPAACAVrmiNR+aNWuWAoGA9u7dq2+++UbHjx/Xnj17FAgE9POf/9zuHgEAAFqtVXd2/vznP2vDhg267rrrwmMDBgzQb37zGyYoAwAAR2nVnZ3GxkZ16dKlyXiXLl3U2NjY5qYAAADs0qqwc+utt+qxxx7T0aNHw2NHjhzRv//7v+u2226zrTkAAIC2alXYWbhwoerq6pSZmam+ffvqn/7pn5SVlaW6ujotWLDA7h4BAABarVVzdtLS0rRz506VlZVp3759sixLAwYM0KhRo+zuDwAAoE1adGdn06ZNGjBggAKBgCTp9ttv16xZs/Tzn/9cQ4cO1fXXX6/333+/XRoFAABojRaFnVdffVUzZsxQQkJCk2M9e/bUT3/6U82fP9+25gAAANqqRWHnk08+0ejRo5s9np+fr/Ly8jY3BQAAYJcWhZ2vv/76vD85P6dz587629/+1uamAAAA7NKisPO9731Pu3fvbvb4p59+Kq/X2+amAAAA7NKisHPnnXfqP/7jP/Ttt982ORYMBlVcXKy7777btuYAAADaqkU/PX/mmWf0hz/8Qf3799ejjz6qa6+9Vi6XSxUVFfrNb36js2fPau7cue3VKwAAQIu1KOwkJydr27Zt+rd/+zcVFRXJsixJksvl0h133KHXXntNycnJ7dIoAABAa7T4pYIZGRlav369jh8/roMHD8qyLPXr10+9evVqj/4AAADapFVvUJakXr16aejQoXb2AgAAYLtWrY0FAAAQKwg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0x4edzMxMuVyuJtvMmTMlSdOmTWtybPjw4VHuGgAAOEWrFwLtKNu3b9fZs2fD+3v27NHtt9+u8ePHh8dGjx6tJUuWhPe7du3aoT0CAADncnzY6dOnT8T+L3/5S/Xt21cjR44Mj7ndbqWkpHR0awAAIAY4/jHWd50+fVrLly/XQw89JJfLFR7fvHmzkpKS1L9/f82YMUM1NTVR7BIAADiJ4+/sfNfbb7+tEydOaNq0aeGxgoICjR8/XhkZGTp06JCeffZZ3XrrrSovL5fb7T5vnVAopFAoFN4PBALt3ToAAIiSmAo7b7zxhgoKCpSamhoemzBhQvjvnJwcDRkyRBkZGVq3bp3Gjh173jqlpaWaN29eu/cLAACiL2bCzpdffqkNGzboD3/4wwXP83q9ysjI0IEDB5o9p6ioSIWFheH9QCCgtLQ023oFnMrn88nv99tWr6KiwrZaANBeYibsLFmyRElJSbrrrrsueN6xY8d0+PBheb3eZs9xu93NPuICTOXz+ZSdfZ2CwVO2124Inba9JgDYJSbCTmNjo5YsWaKpU6eqc+f/33J9fb1KSko0btw4eb1eVVZW6umnn5bH49F9990XxY4B5/H7/QoGT2nYQ8VK8GbaUrNq94fas3aRzpw5Y0s9AGgPMRF2NmzYIJ/Pp4ceeihivFOnTtq9e7fefPNNnThxQl6vV3l5eVq9erXi4+Oj1C3gbAneTCWmX2tLrUBVpS11AKA9xUTYyc/Pl2VZTcbj4uL0zjvvRKEjAAAQK2LqPTsAAAAtRdgBAABGi4nHWAAAXCo7X4ng8XiUnp5uWz1EB2EHAGCEYO0xSS5NnjzZtppxcd21b18FgSfGEXYAAEZoOFUnydKgSU+qT1Z2m+sFqir118Xz5Pf7CTsxjrADADDKlUnptr1eAWZggjIAADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0TpHuwEAAJysoqLC1noej0fp6em21sSFEXYAADiPYO0xSS5NnjzZ1rpxcd21b18FgacDEXYAADiPhlN1kiwNmvSk+mRl21IzUFWpvy6eJ7/fT9jpQIQdAAAu4MqkdCWmXxvtNtAGhB3AJj6fT36/39aaPNsHgLYj7AA28Pl8ys6+TsHgKVvr8mwfANqOsAPYwO/3Kxg8pWEPFSvBm2lLTZ7tA4A9CDuAjRK8mTzbBwCH4aWCAADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0R4edkpISuVyuiC0lJSV83LIslZSUKDU1VXFxccrNzdXevXuj2DEAAHAaR4cdSbr++utVVVUV3nbv3h0+9uKLL2r+/PlauHChtm/frpSUFN1+++2qq6uLYscAAMBJHB92OnfurJSUlPDWp08fSX+/q/Pqq69q7ty5Gjt2rHJycrRs2TKdOnVKK1eujHLXAADAKRwfdg4cOKDU1FRlZWVp4sSJ+uKLLyRJhw4dUnV1tfLz88Pnut1ujRw5Utu2bYtWuwAAwGE6R7uBCxk2bJjefPNN9e/fX19//bWef/55jRgxQnv37lV1dbUkKTk5OeIzycnJ+vLLLy9YNxQKKRQKhfcDgYD9zQMAAEdwdNgpKCgI/33DDTfoBz/4gfr27atly5Zp+PDhkiSXyxXxGcuymoz9o9LSUs2bN8/+hgEAgOM4/jHWd/Xo0UM33HCDDhw4EP5V1rk7POfU1NQ0udvzj4qKilRbWxveDh8+3G49AwCA6IqpsBMKhVRRUSGv16usrCylpKSorKwsfPz06dPasmWLRowYccE6brdbCQkJERsAADCTox9jzZkzR2PGjFF6erpqamr0/PPPKxAIaOrUqXK5XJo9e7ZeeOEF9evXT/369dMLL7yg7t27a9KkSdFuHQAAOISjw85XX32lH//4x/L7/erTp4+GDx+ujz76SBkZGZKkJ554QsFgUD/72c90/PhxDRs2TO+++67i4+Oj3DkAAHAKR4edVatWXfC4y+VSSUmJSkpKOqYhAAAQc2Jqzg4AAEBLEXYAAIDRHP0YCwAAE1VUVNhWy+PxKD093bZ6JiLsAADQQYK1xyS5NHnyZNtqxsV11759FQSeCyDsAADQQRpO1UmyNGjSk+qTld3meoGqSv118Tz5/X7CzgUQdgAA6GBXJqUrMf3aaLdx2WCCMgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDROke7AQAXVlFR4ag6ABBrCDuAQwVrj0lyafLkybbWbQidtrUeADgdYQdwqIZTdZIsDZr0pPpkZbe5XtXuD7Vn7SKdOXOm7c0BQAwh7AAOd2VSuhLTr21znUBVZdubAYAYxARlAABgNMIOAAAwGo+xEBN8Pp/8fr+tNT0ej9LT022tCQBwHsIOHM/n8yk7+zoFg6dsrRsX11379lUQeADAcIQdOJ7f71cweErDHipWgjfTlpqBqkr9dfE8+f1+wg4AGI6wg5iR4M205VdJAIDLC2EHAIAYZ/cb0k2b00jYAQAgRrXXm9ZNm9NI2MFljXWnAMQyu9+0Lpk5p5Gwg8sS604BMIldb1o3FWEHlyXWnQKAywdhB5c11p0CAPOxXAQAADAaYQcAABjN0WGntLRUQ4cOVXx8vJKSknTvvfdq//79EedMmzZNLpcrYhs+fHiUOgYAAE7j6Dk7W7Zs0cyZMzV06FCdOXNGc+fOVX5+vj777DP16NEjfN7o0aO1ZMmS8H7Xrl2j0S6+w86FO/lZNwCgLRwddv785z9H7C9ZskRJSUkqLy/XP//zP4fH3W63UlJSOro9NKO9Fu7kZ90AgNZwdNj5R7W1tZKkxMTEiPHNmzcrKSlJV111lUaOHKlf/OIXSkpKarZOKBRSKBQK7wcCgfZp+DJl98Kd/KwbANAWMRN2LMtSYWGhbrnlFuXk5ITHCwoKNH78eGVkZOjQoUN69tlndeutt6q8vFxut/u8tUpLSzVv3ryOav2yZdfCnfysGwDQFjETdh599FF9+umn+uCDDyLGJ0yYEP47JydHQ4YMUUZGhtatW6exY8eet1ZRUZEKCwvD+4FAQGlpae3TOAAAiKqYCDuzZs3S2rVrtXXrVl199dUXPNfr9SojI0MHDhxo9hy3293sXR8AAGAWR4cdy7I0a9YsrVmzRps3b1ZWVtZFP3Ps2DEdPnxYXq+3AzoEAABO5+j37MycOVPLly/XypUrFR8fr+rqalVXVysYDEqS6uvrNWfOHH344YeqrKzU5s2bNWbMGHk8Ht13331R7h4AADiBo+/svP7665Kk3NzciPElS5Zo2rRp6tSpk3bv3q0333xTJ06ckNfrVV5enlavXq34+PgodAwAAJzG0WHHsqwLHo+Li9M777zTQd0AAIBY5Oiwg45h59uOJd54DABwFsLOZa693nYs8cZjAIAzEHYuc3a/7VjijccAAGch7ECSfW87lnjjMQDAWRz903MAAIC2IuwAAACjEXYAAIDRCDsAAMBoTFCOQXa+F4d34gAATEfYiTHt9V4c3okDADAVYSfG2P1eHN6JAwAwHWEnRtn1XhzeiQMAMB0TlAEAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjMZPzwEAQBN2vmHf4/EoPT3dtnotRdgBAABhwdpjklyaPHmybTXj4rpr376KqAUewg4AAAhrOFUnydKgSU+qT1Z2m+sFqir118Xz5Pf7CTsAAMA5rkxKt+VN/U7ABGUAAGA0wg4AADAaj7Hamc/nk9/vt62enbPjAQC4HBB22pHP51N29nUKBk/ZXrshdNr2mgAAmIiw0478fr+CwVMa9lCxEryZttSs2v2h9qxdpDNnzthSDwAA0xF2OkCCN9O2Ge2Bqkpb6gAAcLlggjIAADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGM2YsPPaa68pKytL3bp10+DBg/X+++9HuyUAAOAARoSd1atXa/bs2Zo7d64+/vhj/ehHP1JBQYF8Pl+0WwMAAFFmRNiZP3++pk+frocffljXXXedXn31VaWlpen111+PdmsAACDKYn7V89OnT6u8vFxPPfVUxHh+fr62bdt23s+EQiGFQqHwfm1trSQpEAjY2lt9fb0k6Zsv9+tMKGhLzUDVl5Kk2iMH1KWzy3H12qMmPdKjk2rSIz06qWZM9Fj996cs9fX1tv939lw9y7IufKIV444cOWJJsv7yl79EjP/iF7+w+vfvf97PFBcXW5LY2NjY2NjYDNgOHz58wawQ83d2znG5ItOnZVlNxs4pKipSYWFheL+xsVHffPONevfu3exnnCwQCCgtLU2HDx9WQkJCtNuJCVyzluF6tRzXrOW4Zi13uV8zy7JUV1en1NTUC54X82HH4/GoU6dOqq6ujhivqalRcnLyeT/jdrvldrsjxq666qr2arHDJCQkXJb/srcF16xluF4txzVrOa5Zy13O16xnz54XPSfmJyh37dpVgwcPVllZWcR4WVmZRowYEaWuAACAU8T8nR1JKiws1JQpUzRkyBD94Ac/0KJFi+Tz+fTII49EuzUAABBlRoSdCRMm6NixY3ruuedUVVWlnJwcrV+/XhkZGdFurUO43W4VFxc3eTSH5nHNWobr1XJcs5bjmrUc1+zSuCzrYr/XAgAAiF0xP2cHAADgQgg7AADAaIQdAABgNMIOAAAwGmHHQOvWrdOwYcMUFxcnj8ejsWPHRrulmBAKhTRo0CC5XC7t2rUr2u04VmVlpaZPn66srCzFxcWpb9++Ki4u1unTp6PdmqO89tprysrKUrdu3TR48GC9//770W7JsUpLSzV06FDFx8crKSlJ9957r/bv3x/ttmJGaWmpXC6XZs+eHe1WHIuwY5j//d//1ZQpU/Tggw/qk08+0V/+8hdNmjQp2m3FhCeeeOKirxyHtG/fPjU2Nup3v/ud9u7dq1deeUW//e1v9fTTT0e7NcdYvXq1Zs+erblz5+rjjz/Wj370IxUUFMjn80W7NUfasmWLZs6cqY8++khlZWU6c+aM8vPzdfLkyWi35njbt2/XokWL9P3vfz/arTibLatxwhEaGhqs733ve9Z///d/R7uVmLN+/XorOzvb2rt3ryXJ+vjjj6PdUkx58cUXraysrGi34Rg333yz9cgjj0SMZWdnW0899VSUOootNTU1liRry5Yt0W7F0erq6qx+/fpZZWVl1siRI63HHnss2i05Fnd2DLJz504dOXJEV1xxhW688UZ5vV4VFBRo79690W7N0b7++mvNmDFDb731lrp37x7tdmJSbW2tEhMTo92GI5w+fVrl5eXKz8+PGM/Pz9e2bdui1FVsqa2tlST+nbqImTNn6q677tKoUaOi3YrjEXYM8sUXX0iSSkpK9Mwzz+hPf/qTevXqpZEjR+qbb76JcnfOZFmWpk2bpkceeURDhgyJdjsx6fPPP9eCBQtYnuX/8fv9Onv2bJOFiJOTk5ssWIymLMtSYWGhbrnlFuXk5ES7HcdatWqVdu7cqdLS0mi3EhMIOzGgpKRELpfrgtuOHTvU2NgoSZo7d67GjRunwYMHa8mSJXK5XPr9738f5W/RsS71mi1YsECBQEBFRUXRbjnqLvWafdfRo0c1evRojR8/Xg8//HCUOncml8sVsW9ZVpMxNPXoo4/q008/1f/8z/9EuxXHOnz4sB577DEtX75c3bp1i3Y7McGItbFM9+ijj2rixIkXPCczM1N1dXWSpAEDBoTH3W63rrnmmstuYuSlXrPnn39eH330UZN1ZYYMGaIHHnhAy5Yta882HeVSr9k5R48eVV5eXnjxXfydx+NRp06dmtzFqampaXK3B5FmzZqltWvXauvWrbr66quj3Y5jlZeXq6amRoMHDw6PnT17Vlu3btXChQsVCoXUqVOnKHboPISdGODxeOTxeC563uDBg+V2u7V//37dcsstkqSGhgZVVlZeNouinnOp1+y//uu/9Pzzz4f3jx49qjvuuEOrV6/WsGHD2rNFx7nUayZJR44cUV5eXvju4RVXcJP4nK5du2rw4MEqKyvTfffdFx4vKyvTv/zLv0SxM+eyLEuzZs3SmjVrtHnzZmVlZUW7JUe77bbbtHv37oixBx98UNnZ2XryyScJOudB2DFIQkKCHnnkERUXFystLU0ZGRl66aWXJEnjx4+PcnfOlJ6eHrF/5ZVXSpL69u3L/7NsxtGjR5Wbm6v09HT96le/0t/+9rfwsZSUlCh25hyFhYWaMmWKhgwZEr7z5fP5mNfUjJkzZ2rlypX64x//qPj4+PBdsZ49eyouLi7K3TlPfHx8k/lMPXr0UO/evZnn1AzCjmFeeuklde7cWVOmTFEwGNSwYcO0adMm9erVK9qtwRDvvvuuDh48qIMHDzYJhJZlRakrZ5kwYYKOHTum5557TlVVVcrJydH69esvuzusl+r111+XJOXm5kaML1myRNOmTev4hmAcl8X/OgEAAIPxoB0AABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAo/0flseG1iGnzOoAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sns.histplot(reg_results.resid)" ] }, { "cell_type": "markdown", "id": "17293ed3", "metadata": {}, "source": [ "The residuals look roughly normally distributed. It looks like the normality assumption has been met" ] }, { "cell_type": "code", "execution_count": null, "id": "05988e86", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 5 }