Birth Weight | Gestational Days | Maternal Age | Maternal Height | Maternal Pregnancy Weight | Maternal Smoker |
---|---|---|---|---|---|

120 | 284 | 27 | 62 | 100 | False |

113 | 282 | 33 | 64 | 135 | False |

128 | 279 | 28 | 64 | 115 | True |

108 | 282 | 23 | 67 | 125 | True |

136 | 286 | 25 | 62 | 93 | False |

138 | 244 | 33 | 62 | 178 | False |

132 | 245 | 23 | 65 | 140 | False |

120 | 289 | 25 | 62 | 125 | False |

143 | 299 | 30 | 66 | 136 | True |

140 | 351 | 27 | 68 | 120 | False |

... (1164 rows omitted)

" ], "text/plain": [ "Birth Weight | Gestational Days | Maternal Age | Maternal Height | Maternal Pregnancy Weight | Maternal Smoker\n", "120 | 284 | 27 | 62 | 100 | False\n", "113 | 282 | 33 | 64 | 135 | False\n", "128 | 279 | 28 | 64 | 115 | True\n", "108 | 282 | 23 | 67 | 125 | True\n", "136 | 286 | 25 | 62 | 93 | False\n", "138 | 244 | 33 | 62 | 178 | False\n", "132 | 245 | 23 | 65 | 140 | False\n", "120 | 289 | 25 | 62 | 125 | False\n", "143 | 299 | 30 | 66 | 136 | True\n", "140 | 351 | 27 | 68 | 120 | False\n", "... (1164 rows omitted)" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "births = Table.read_table(path_data + 'baby.csv')\n", "births" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One of the aims of the study was to see whether maternal smoking was associated with birth weight. Let's see what we can say about the two variables.\n", "\n", "We'll start by selecting just `Birth Weight` and `Maternal Smoker`. There are 715 non-smokers among the women in the sample, and 459 smokers." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "smoking_and_birthweight = births.select('Maternal Smoker', 'Birth Weight')" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "Maternal Smoker | count |
---|---|

False | 715 |

True | 459 |

"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"smoking_and_birthweight.hist('Birth Weight', group = 'Maternal Smoker')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The distribution of the weights of the babies born to mothers who smoked appears to be based slightly to the left of the distribution corresponding to non-smoking mothers. The weights of the babies of the mothers who smoked seem lower on average than the weights of the babies of the non-smokers. \n",
"\n",
"This raises the question of whether the difference reflects just chance variation or a difference in the distributions in the larger population. Could it be that there is no difference between the two distributions in the population, but we are seeing a difference in the samples just because of the mothers who happened to be selected?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Hypotheses\n",
"We can try to answer this question by a test of hypotheses. The chance model that we will test says that there is no underlying difference in the populations; the distributions in the samples are different just due to chance. \n",
"\n",
"Formally, this is the null hypothesis. We are going to have to figure out how to simulate a useful statistic under this hypothesis. But as a start, let's just state the two natural hypotheses.\n",
"\n",
"**Null hypothesis:** In the population, the distribution of birth weights of babies is the same for mothers who don't smoke as for mothers who do. The difference in the sample is due to chance.\n",
"\n",
"**Alternative hypothesis:** In the population, the babies of the mothers who smoke have a lower birth weight, on average, than the babies of the non-smokers."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Test Statistic\n",
"The alternative hypothesis compares the average birth weights of the two groups and says that the average for the mothers who smoke is smaller. Therefore it is reasonable for us to use the difference between the two group means as our statistic. \n",
"\n",
"We will do the subtraction in the order \"average weight of the smoking group $-$ average weight of the non-smoking group\". Small values (that is, large negative values) of this statistic will favor the alternative hypothesis. \n",
"\n",
"The observed value of the test statistic is about $-9.27$ ounces."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

"
],
"text/plain": [
"Maternal Smoker | Birth Weight average\n",
"False | 123.085\n",
"True | 113.819"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"means_table = smoking_and_birthweight.group('Maternal Smoker', np.average)\n",
"means_table"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"-9.266142572024918"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"means = means_table.column(1)\n",
"observed_difference = means.item(1) - means.item(0)\n",
"observed_difference"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We are going compute such differences repeatedly in our simulations below, so we will define a function to do the job. The function takes two arguments:\n",
"\n",
"- the name of the table of data\n",
"- the label of the column that contains the Boolean variable for grouping\n",
"\n",
"It returns the difference between the means of the `True` group and the `False` group.\n",
"\n",
"You will soon see why we are specifying the two arguments. For now, just check that the function returns what it should."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"def difference_of_means(table, group_label):\n",
" \"\"\"Takes: name of table,\n",
" column label that indicates the group to which the row belongs\n",
" Returns: Difference of mean birth weights of the two groups\"\"\"\n",
" reduced = table.select('Birth Weight', group_label)\n",
" means_table = reduced.group(group_label, np.average)\n",
" means = means_table.column(1)\n",
" return means.item(1) - means.item(0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To check that the function is working, let's use it to calculate the observed difference between the mean birth weights of the two groups in the sample."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"-9.266142572024918"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"difference_of_means(births, 'Maternal Smoker')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That's the same as the value of `observed_difference` calculated earlier."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Predicting the Statistic Under the Null Hypothesis\n",
"\n",
"To see how the statistic should vary under the null hypothesis, we have to figure out how to simulate the statistic under that hypothesis. A clever method based on *random permutations* does just that.\n",
"\n",
"If there were no difference between the two distributions in the underlying population, then whether a birth weight has the label `True` or `False` with respect to maternal smoking should make no difference to the average. The idea, then, is to shuffle all the labels randomly among the mothers. This is called *random permutation*. \n",
"\n",
"Shuffling ensures that the count of `True` labels does not change, and nor does the count of `False` labels. This is important for the comparability of the simulated differences of means and the original difference of means. We will see later in the course that the sample size affects the variability of a sample mean.\n",
"\n",
"Take the difference of the two new group means: the mean weight of the babies whose mothers have been randomly labeled smokers and the mean weight of the babies of the remaining mothers who have all been randomly labeled non-smokers. This is a simulated value of the test statistic under the null hypothesis.\n",
"\n",
"Let's see how to do this. It's always a good idea to start with the data. We have reduced the table to have just the columns that we need."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

\n",
"

"
],
"text/plain": [
"Shuffled Label | Birth Weight average\n",
"False | 119.277\n",
"True | 119.752"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"shuffled_only = original_and_shuffled.select('Birth Weight','Shuffled Label')\n",
"shuffled_group_means = shuffled_only.group('Shuffled Label', np.average)\n",
"shuffled_group_means"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The averages of the two randomly selected groups are quite a bit closer than the averages of the two original groups. We can use our function `difference_of_means` to find the two differences."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.4747109100050153"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"difference_of_means(original_and_shuffled, 'Shuffled Label')"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"-9.266142572024918"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"difference_of_means(original_and_shuffled, 'Maternal Smoker')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"But could a different shuffle have resulted in a larger difference between the group averages? To get a sense of the variability, we must simulate the difference many times. \n",
"\n",
"As always, we will start by defining a function that simulates one value of the test statistic under the null hypothesis. This is just a matter of collecting the code that we wrote above. \n",
"\n",
"The function is called `one_simulated_difference_of_means`. It takes no arguments, and returns the difference between the mean birth weights of two groups formed by randomly shuffling all the labels."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"def one_simulated_difference_of_means():\n",
" \"\"\"Returns: Difference between mean birthweights\n",
" of babies of smokers and non-smokers after shuffling labels\"\"\"\n",
" \n",
" # array of shuffled labels\n",
" shuffled_labels = births.sample(with_replacement=False).column('Maternal Smoker')\n",
" \n",
" # table of birth weights and shuffled labels\n",
" shuffled_table = births.select('Birth Weight').with_column(\n",
" 'Shuffled Label', shuffled_labels)\n",
" \n",
" return difference_of_means(shuffled_table, 'Shuffled Label') "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the cell below a few times to see how the output changes."
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"-0.058299434770034964"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"one_simulated_difference_of_means()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Permutation Test\n",
"Tests based on random permutations of the data are called *permutation tests*. We are performing one in this example. In the cell below, we will simulate our test statistic – the difference between the average birth weight of the two randomly formed groups – many times and collect the differences in an array. "
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"differences = make_array()\n",
"\n",
"repetitions = 5000\n",
"for i in np.arange(repetitions):\n",
" new_difference = one_simulated_difference_of_means()\n",
" differences = np.append(differences, new_difference) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The array `differences` contains 5,000 simulated values of our test statistic: the difference between the mean weight in the smoking group and the mean weight in the non-smoking group, when the labels have been assigned at random. \n",
"\n",
"## Conclusion of the Test\n",
"The histogram below shows the distribution of these 5,000 values. It is the empirical distribution of the test statistic simulated under the null hypothesis. This is a prediction about the test statistic, based on the null hypothesis."
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Observed Difference: -9.266142572024918\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAbEAAAExCAYAAADyYJxpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde1xM+f8H8NcoJRVT1FAUqkW57cplpcidKCK5rbXusWtbq017wbpscluXTZvr1rq2ZOWWayjJWsvadVn5uiQl6UYjRc3vD4+ZnzFTTZrMTF7Px8Njd875nPN5z5nTvObcBbm5uRIQERHpoBqaLoCIiOhNMcSIiEhnMcSIiEhnMcSIiEhnMcSIiEhnMcSIiEhnMcQ0xM/PD0KhEPHx8XLDhUIhPDw8qqzf4OBgpf3SS9Lls3XrVk2XUioPDw8IhULcvXtX06W8Fcr+Jrgel23r1q0QCoUIDg7WdCkAqvbzqtYhJhQK5f6Zm5vD1tYW/fr1w+bNm1FcXKzpEtVOF76EXyetuaw/uPj4+CoPeG2h7SHVunVrCIVCWFtbIz09XWmb8ePHa1XIVLd1TNtCSpP0NV3A2xAYGAgAKC4uxu3bt7F//34kJSXh5MmTiIiI0HB18v744w8YGRlV2fwnT56MoUOHolGjRlXWB70bxGIxFi5ciNDQUE2XQlquKr933okQCwoKknt95coV9OrVC3v37kViYiK6dOmiocoUvffee1U6/3r16qFevXpV2ge9G+zs7LB9+3ZMmTIFbdq00XQ5pMWq8nunWu9OLI2TkxNcXFwAABcuXADw/7sS/Pz8cP36dYwZMwbNmjWDUCjE5cuXZdPu3bsXXl5eaNKkCSwtLfHBBx9g3rx5ePz4sdK+Tp48if79+8PKygpNmjTBqFGj8N9//5VaW2m7M4qLixEZGYn+/fvD1tYWIpEIbdq0wcSJE3Hx4kUAL3dDhYSEAACmT58utytVumuqrH3Tp0+fho+PD5o2bQpLS0u0bdsWgYGByMzMVGj76jG9vXv3okePHmjYsCGaNGmCTz75BPfv3y/1ParTm9Zx6dIl2S/Dxo0bw8vLC+fOnSuzr4yMDMyePRsffPABRCIRbG1tMWTIEJw6dUqh7au7e86dOwdvb2/Y2tpCKBQiNze31D6EQiHOnDkDAGjbtq3s82vdurXS9ps3b0aXLl0gEong4OCAGTNmlDr/itSvinnz5qGkpATffvutytOUtbtOG49zrV+/HkKhEIsXL1Y6/vHjx7CysoKTk5Ps8MSru/RjY2PRu3dv2d//uHHjcPv2baXzysjIQEBAANq2bQtLS0s0bdoUw4cPR0JCglw7Pz8/TJ8+HQAQEhIi93eubNldvnwZw4cPh42NDRo2bIj+/fsjKSlJaQ0lJSWIjIxE3759YWNjA5FIhA8//BArVqxAUVGRQvv4+Hj4+vrCyckJlpaWsLe3R/fu3fHNN99AIvn/OxqW9tmqOn1Z3oktsYq4ffs2+vTpg+bNm2PEiBHIy8tD7dq1AQBffvklNm7cCGtrawwcOBBCoRB//vknVq5ciSNHjuDw4cMwNTWVzWvv3r345JNPULNmTQwePBhWVlZISkpC79690apVK5VrKioqwqhRo3Ds2DE0aNAAQ4YMgZmZGVJTUxEfHw87Ozu8//77GDVqFADgzJkzGDBggNwXX926dcvsY/PmzZg5cyaMjIzg5eWFBg0a4Ny5cwgPD8eBAwdw6NAhNG7cWGG6jRs34tChQxgwYABcXFzw559/Ys+ePfjnn39w5swZGBoaqvw+K6MidZw7dw6DBw9GYWEhBg0aBDs7O1y5cgWDBg2Cm5ub0vlfuXIFQ4YMQWZmJnr06IEBAwYgOzsbBw4cwODBg7F69Wp89NFHCtP98ccfWLFiBbp06YKxY8ciPT0denp6pb6PwMBAbNu2Dffu3cPUqVNln5uyz2/u3Lk4ceIE+vXrB3d3d8THxyMyMhI3b97EwYMH1VJ/WXr37g13d3fExcUhNjYW/fr1q9D0umDEiBGYP38+fv31VwQEBCh8djt27MDTp08xY8YMhXH79u3DsWPHMGjQILi6uuLy5cv4/fffER8fjyNHjsDOzk7W9u7du+jfvz/S0tLg4uICb29vPHjwAL///juOHTuGlStXYuzYsQBe/ljNy8vDwYMH4eLigq5du8rmY2NjI1fDpUuXsHr1anTq1Aljx45FamoqYmJi4OXlhdOnT6N58+ayti9evMCYMWMQGxsLe3t7DB06FIaGhjhz5gzmz5+PU6dOYffu3dDXfxkbR44cga+vL0xNTdG/f39YW1sjNzcX//vf/xAeHo7vv/9e1laZyk4v9U6G2LVr12S/dj/44AO5cUlJSZg5cybmzJkjN3znzp3YuHEjBg4ciPXr18sdt1q6dCkWLVqE4OBg/PDDDwCA/Px8+Pv7QyAQ4MCBA3B2dpa1/+6777BmzRqV6w0JCcGxY8fQvXt3bNu2TRaqwMstNOmW0ujRo5GSkoIzZ87Aw8MDo0ePVmn+KSkpCAwMRO3atXHs2DG0bNlSNm7hwoVYtmwZvvzyS0RFRSlMe+LECZw6dQotWrSQDZs4cSJ27dqFAwcOwNvbW+X3WRmq1iGRSPDpp5+ioKAAERER8PLykrVfv349AgICFOZdXFyMjz/+GHl5edi3b5/cl8aDBw/Qs2dPBAQEoG/fvrC0tJSbNi4uDitXrsS4ceNUeh9BQUFISEjAvXv34OfnB1tb21LbXrhwAWfPnoW1tTWAl19CgwYNQmJiIv7880/ZOleZ+suzYMECuLm5Yc6cOejVq5dKXzqalJCQUOrJECkpKQrDTE1N4evriw0bNiA2NlZhK/KXX36Bvr6+LGBeFRsbi507d6Jv376yYWvWrMF3332HgIAAREdHy4Z/8cUXSEtLw+zZszF79mzZ8E8//RS9evVCQEAAevTogUaNGmHgwIGyEOvatavC4ZJXHT58GOHh4fD19ZUN27x5M7744guEh4djxYoVsuE//vgjYmNjMWnSJCxevFgWyiUlJfjiiy8QERGBDRs2YOrUqQCAyMhISCQS7Nu3D23btpXrNzs7u9x1obLTS70TuxODg4MRHByMhQsXYtKkSXB3d0dBQQEGDhwo260oZWlpKTsR5FVr166Fnp4e1qxZo3DixcyZM1GvXj25L/mDBw8iJycH3t7ecgEGAF999RXq1KmjUu3FxcXYsGEDDA0NsWrVKrkAAwA9PT00aNBApXmVJioqCkVFRZgwYYJcgAFAQEAAGjZsiCNHjiAtLU1h2ilTpsgFBwB8/PHHAIC//vqrUnVVhKp1nDt3DsnJyejUqZNcgAHAhAkT0KxZM4V5HzlyBDdv3sSECRPkAgAAGjRogM8++wzPnj3D3r17FaZt1aqVygFWUV999ZUswABAX18fY8aMASD/nitTf3latWqFMWPG4MaNG9i8efMbvpO358yZMwgJCVH6b/v27UqnmThxIgAovL+kpCRcvXoV/fr1g5WVlcJ0bm5ucgEGvNwV2KhRI5w4cUL293T//n2cOHECVlZWmDlzplx7JycnjB8/HoWFhdi5c2eF3++HH34oF2AAMGbMGOjr68utIyUlJfj5559hYWGB4OBgua3KGjVqYP78+RAIBHI11KjxMj5e/04CAHNz83Jrq+z0Utr9s0lNpMeJBAIBTE1N0bZtW/j4+Cj9cmnVqpXCLrCCggJcvnwZZmZm+Pnnn5X2YWBggPT0dGRnZ8Pc3Bx///03ACiEJPDy112bNm0U9nUrc+PGDeTl5aFt27Zl/iqvDGmtynalGRoaonPnztizZw8uX76s8Mfarl07hWmkX6xlHftRN1XrKOtzqVGjBjp37oxbt27JDZceK0tNTVX6K17a/saNGwrjXv8Bo06qvufK1K+Kb775BtHR0Vi8eDGGDx9e7q5rTQoMDCx1yyU+Ph6DBg1SGN6iRQt07doVJ06cwJ07d9CkSRMA/x9qEyZMUDo/ZeuYvr4+OnXqhNTUVNnfk/SYe+fOnWFgYKAwTffu3REaGipbdytC2TpSs2ZNWFpayq0jN2/eRFZWFpo2bYqlS5cqnZeRkRGSk5Nlr4cPH46YmBj07NkTQ4YMgaurKzp06KDy91Rlp5d6J0KsIl+mynan5OTkQCKRIDs7WxaIpcnPz4e5ubnsRA8LCwuV+1EmLy8PAJT+0lMXaa2l1SQSieTavUrZFqX0V5yq1+FJf5GVlJSU2kY6Ttr2Tet4k88lOzsbABATE4OYmJhSaxSLxSrNT11Ufc+VqV8VIpEIM2bMwA8//IDly5dj/vz5bzQfbTZp0iQkJCQgIiICc+fORU5ODvbu3YtmzZqhe/fuSqcp7bOXrnvSdbEyf3/lKW2Pj56entJ15Pbt2+V+x0kNHDgQu3fvxpo1a7B9+3bZ5UqOjo4IDAxU2NOh7uml3okQqwiBQKAwTLoiODo6IjExUaX5SKdRdmYfADx8+FCl+Uh/1ZZ2Uak6SGstraaMjAy5dlXVf05OTqltpH9klf2V/yafi3SayMhIeHp6Vqg/ZevT21aZ+lX12WefISIiAuHh4aVumQAvl0dpP26kP9i0kYeHB6ysrLBlyxYEBQVh27ZtePbsGcaNG1fqZ1za35N03ZN+Lpr++3t13v369cOOHTtUnq5nz57o2bMnCgoKcOHCBRw7dgwbN27EuHHjFI6/VsX0wDtyTKyyTExM4OjoiOTkZGRlZak0jfRApfQEklc9efJE7rT9srz33nuoW7curl27hnv37pXbvqJbQa/Wquz03MLCQtnuqNcPvqqL9EzN0k77fXVcRc7qVKasz6WkpERpDR06dAAAnD17tlJ9q+rVA+rq8DbqNzIywrfffovCwkJ8//33pbYTCoVITU1VOk56qYg20tfXx8cff4zMzEzs378fERERMDQ0LPPkKWXr2IsXL2R/T9Jr66T/PXfunNLT2KWXQLy6a/BN/s7LIv2euXDhgtIaymNkZISuXbti3rx5WLBgASQSicIZslU1PUNMRdOnT8fz588xbdo0pVsMT548wZ9//il7PWDAAAiFQkRHR8sNB4AlS5aovGtAT08PkyZNQmFhIfz9/VFQUCA3vri4GA8ePJC9ll5QWNoXhTLDhw+HgYEBNm7cqHBcZMWKFUhLS0OfPn3QsGFDledZEV26dEHTpk3x77//IjIyUmH85cuXsWXLFujr6yscpK6oTp06wcHBAefOnVM4kWHjxo0Kx8OAl59ls2bNsHnz5lL/sP7++2/Z1mJlST9DVX60qOJt1T9ixAi0a9cO0dHRuHTpktI2HTp0QGpqKo4cOSI3PCIiotzr9DRt3LhxqFmzJr7++mvcuHEDXl5eZV7Ae/r0aRw+fFhuWFhYGFJTU+Hu7i47RGBtbY2ePXvi/v37WLVqlVz7a9euYdOmTTA0NMTw4cNlw9/k77ws+vr6mDp1KjIzMzFr1iw8ffpUoU1WVpbcj++TJ08qbSfdcqxVq1aZfVZ2elntKrUijB49Gn///TfWrVuHdu3aoWfPnrCxsUFeXh5SUlKQmJgId3d3bNu2DcDLrbdVq1bhk08+gYeHB4YMGQIrKyucPXsWV69eRZcuXVTeNfnVV1/h4sWLOH78OD744AP069cPZmZmSEtLQ3x8PMaMGSM7WN2tWzfUqFEDP//8M3JycmT72SdPnlzqrjgbGxuEhIRg5syZcHd3x+DBgyESiXDu3DmcOXMG1tbWWL58uRqWonJ6enpYt24dhg4dihkzZmD79u1wdnZGzZo1cePGDRw+fBjFxcVYsmQJmjZtWqm+BAIB1qxZgyFDhuCTTz6Ru04sLi4OvXr1wrFjx+SmqVmzJrZs2QJvb2+MGjUKzs7OaNu2LYyNjXH//n1cvnwZycnJOH36dIXOqiqNu7s79uzZg88//xxeXl4wNjZG3bp1MXny5Dea39uqXyAQYOHChRg4cKDSHwMAMGPGDBw7dgxjxozB4MGDYWFhgUuXLuHSpUvo27evwpe+NhGJRBg4cCD27NkD4OX9IcvSv39/jB49Gp6enmjSpAkuX76MY8eOwdzcHMuWLZNru2LFCvTr1w+LFi3C6dOn0aFDB9l1YgUFBVi1apXcLZs6duwIExMTREdHw8DAAI0aNYJAIICvr6/CtWKqCggIwNWrVxEZGYkjR47Azc0N1tbWePToEW7fvo2kpCRMnDhRtuX47bffIiUlBS4uLrCxsUGtWrVw5coVHD9+HObm5rKzg0tT2emlGGIVsGTJEvTp0wcbN25EQkICcnJyULduXVhZWWHChAnw8fGRa+/l5YXdu3cjJCQEe/fuhYGBAbp06YKjR4/ixx9/VDnEDAwMEBUVhYiICGzfvh2//fYbXrx4AZFIBBcXF/Tv31/W1t7eHhs3bsSqVauwZcsW2ZZbeWeNffLJJ2jWrBnWrFmDAwcOQCwWo2HDhpg8eTJmzZpVpScoAC9/oSckJOCnn35CXFwcNmzYgOLiYlhYWMDT0xNTpkxBx44d1dJX586dcejQISxYsADHjx/H8ePH0b59e+zfvx/Hjx9XCDHg5fHQM2fOICwsDAcPHsT27dshkUggEonQokULfPbZZ3BwcFBLfWPGjMH9+/cRFRWF0NBQPH/+HI0bN37jEHub9Xft2hUDBgwodYuva9eu2LlzJxYvXoyYmBi5v4m9e/dqdYgBLz+bPXv2wNHREZ07dy6z7cCBAzFu3DgsW7YMsbGxqFmzJry8vDB37lyFSzlsbW1x8uRJWdukpCQYGxvDxcUFM2bMgKurq1z7unXrYuvWrQgODkZ0dDTy8/MBvFy33zTE9PX1ERkZid27d2Pr1q04evSo7ES1xo0b44svvsCIESNk7b/88kscOHAAFy9elB2KsLKygp+fH6ZNm1bufRIrO72UIDc3V7V7exARveOWL1+OBQsWYNmyZbLrx14XHByMkJAQhIaGqnzDAXpzPCZGRKSC/Px8rF+/HnXq1Kn0sVlSH+5OJCIqw6FDh3Dx4kUcPXoUDx48wNy5c+XukUqaxRAjIipDTEwMtm/fDktLS/j7+2PGjBmaLolewWNiRESks3hMjIiIdBZDjIiIdBZDjIiIdBZDjBS8+rgFUj8u36rF5Vv1tGkZM8SIiEhnMcSIiEhnMcSIiEhnMcSIiEhnMcSIiEhnMcSIiEhnMcSIiEhnMcSIiEhnaewu9uvXr8fmzZtx7949AECLFi0wa9Ys9O3bFwDg5+eH7du3y03j7Oys9Km7RG9LRmYOHuU8rtQ88sVFuHLjbqXmUd+sDkQWZpWaB1F1oLEQs7Kywvfffw87OzuUlJRg+/btGD16NE6ePIlWrVoBALp3747w8HDZNAYGBpoqlwgA8CjnMeav3lqpeYjFYhgbG1dqHnNmjGaIEUGDIebh4SH3+rvvvsPGjRtx/vx5WYgZGhpCJBJpojwiItIBWnFMrLi4GLt374ZYLEbHjh1lw8+ePQt7e3u0b98eM2bMQGZmpgarJCIibaPRJztfuXIFffr0wbNnz2BsbIwtW7bAyckJANCrVy8MGjQItra2SElJwcKFC+Hp6YmTJ0/C0NBQk2UTEZGW0OiTnYuKipCamoq8vDzExMQgIiIC+/fvh6Ojo0Lb9PR0tG7dGps2bYKnp2ep89SmuytT9ZOdX4QFq7dpugx8N2MUzE14jJiqPwcHhzLHa3RLzMDAAM2aNQMAvP/++/jrr7+wdu1a/PTTTwptGzZsCCsrK9y6davMeZb3hql8ycnJXI6luHLjbqVPylDHiR0mxiZwcLCt1DyqK66/VU+blrFWHBOTKikpQVFRkdJxWVlZSE9P54keREQko7EtsXnz5qFPnz6wtrZGfn4+du3ahYSEBERFRSE/Px+LFy+Gp6cnRCIRUlJSMH/+fFhYWGDgwIGaKpmIiLSMxkIsIyMDkydPxsOHD1GnTh04OTlh165d6NmzJwoKCnD16lXs2LEDeXl5EIlEcHV1xebNm2FqaqqpkomISMtoLMTCwsJKHWdkZITo6Oi3WA0REekirTomRkREVBEMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkaC7H169ejS5cuaNy4MRo3bozevXvj8OHDsvESiQTBwcFo0aIFGjRoAA8PD1y7dk1T5RIRkRbSWIhZWVnh+++/x6lTpxAXFwc3NzeMHj0a//77LwBg1apVCA0NRUhICE6cOAELCwsMGTIET5480VTJRESkZTQWYh4eHujduzeaNWsGe3t7fPfddzAxMcH58+chkUgQFhYGf39/eHl5wdHREWFhYcjPz8euXbs0VTIREWkZrTgmVlxcjN27d0MsFqNjx464e/cuMjIy0KNHD1kbIyMjdOnSBefOndNgpUREpE30Ndn5lStX0KdPHzx79gzGxsbYsmULnJycZEFlYWEh197CwgLp6emaKJWIiLSQRkPMwcEB8fHxyMvLQ0xMDPz8/LB//37ZeIFAINdeIpEoDHtdcnJyldT6ruFyVC5fXASxWFzp+VR2HvnifH5GZeCyqXpvaxk7ODiUOV6jIWZgYIBmzZoBAN5//3389ddfWLt2LWbNmgUAePjwIRo1aiRr/+jRI4Wts9eV94apfMnJyVyOpbhy4y6MjY0rNQ+xWFzpeZgYm8DBwbZS86iuuP5WPW1axlpxTEyqpKQERUVFsLW1hUgkQlxcnGzcs2fPcPbsWXTq1EmDFRIRkTZReUvszJkzaN68OerXr690fFZWFq5fvw4XFxeV5jdv3jz06dMH1tbWsrMOExISEBUVBYFAAD8/PyxfvhwODg6wt7fHsmXLYGxsjGHDhqlaMhERVXMqh9igQYMQHh4OHx8fpeNPnTqFiRMnIjs7W6X5ZWRkYPLkyXj48CHq1KkDJycn7Nq1Cz179gQAfP755ygoKEBAQAByc3PRvn17REdHw9TUVNWSiYiomlM5xCQSSZnji4qKUKOG6nsnw8LCyhwvEAgQFBSEoKAgledJRETvljJD7PHjx8jLy5O9zs7Oxr179xTa5ebmYvfu3WjYsKH6KyQiIipFmSG2du1aLFmyBED5W0YSiQTfffed+iskIiIqRZkh1r17d9SqVQsSiQTz58+Ht7c3WrduLddGIBCgdu3aeP/99+Hs7FylxRIREb2qzBDr3LkzOnfuDAAoLCzEoEGD4OTk9FYKIyIiKo/KJ3bMnj27KusgIiKqsFJDbPv27QCAESNGQCAQyF6XZ+TIkeqpjIiIqBylhti0adMgEAgwdOhQGBgYYNq0aeXOTCAQMMSIiOitKTXE/v77bwAv72/46msiTcjIzMGjnMeaLgOFRUWaLoGIXlFqiNnY2JT5muhtepTzGPNXb9V0GfAfP0TTJRDRK7TqBsBEREQVUaFHsZw8eRIRERG4c+cOcnJyFG5FJRAIcOnSJbUWSEREVBqVQywsLAzffPMN6tevD2dnZ7Rs2bIq6yIiIiqXyiEWGhoKFxcX7N69W3ayBxFphkDw8gGd2qC+WR2ILMw0XQa9o1QOsaysLHz55ZcMMCItkPfkKVZu2qPpMgAAc2aMZoiRxqh8Yke7du2QkpJSlbUQERFViMohtmjRImzbtg2nT5+uynqIiIhUpvLuxODgYNSpUweDBw+GnZ0dGjduDD09Pbk2AoEAUVFRai+SiIhIGZVD7Pr16xAIBGjUqBEKCwtx8+ZNhTYCgUCtxREREZVF5RD7559/qrIOIiKiCuMdO4iISGepvCV27949ldo1btz4jYshIiKqCJVDrE2bNiod88rOzlZpfitWrMC+fftw8+ZNGBgYwNnZGXPnzoWjo6OsjZ+fn8JzzJydnXHs2DFVyyYiompM5RD76aefFEKsuLgYd+/exY4dO2BpaYmJEyeq3HFCQgImTJiADz74ABKJBD/88AMGDx6Mc+fOwczs/y+c7N69O8LDw2WvebE1ERFJqRxio0ePLnWcv78/evTogfz8fJU7jo6OlnsdHh4OGxsbJCUloX///rLhhoaGEIlEKs+XiIjeHWo5scPExASjR4/G2rVr33ge+fn5KCkpgVAolBt+9uxZ2Nvbo3379pgxYwYyMzMrWy4REVUTFXoUS1lq1qyJ9PT0N55+9uzZaN26NTp27Cgb1qtXLwwaNAi2trZISUnBwoUL4enpiZMnT8LQ0FDpfJKTk9+4Bvp/2rYc88VFEIvFmi4DL4pfqKWOys5DXXWoQ744X+vWF22rpzp6W8vYwcGhzPFqCbF//vkHP//8M5o3b/5G03/99ddISkpCbGys3F1Ahg4dKvt/JycntGvXDq1bt8bhw4fh6empdF7lvWEqX3JystYtxys37sLY2FjTZUBfT7/SdYjF4krPQx11qIuJsQkcHGw1XYaMNq6/1Y02LeNKn52Yl5eHx48fw8TEBKGhoRUuICgoCNHR0di3bx+aNGlSZtuGDRvCysoKt27dqnA/RERU/agcYi4uLgohJhAIIBQK0axZMwwdOlTheFZ5AgMDER0djf379+O9994rt31WVhbS09N5ogcREQGo4JOd1WnWrFnYuXMntmzZAqFQiIyMDACAsbExTExMkJ+fj8WLF8PT0xMikQgpKSmYP38+LCwsMHDgQLXWQkREukltJ3ZU1IYNGwAAXl5ecsMDAwMRFBQEPT09XL16FTt27EBeXh5EIhFcXV2xefNmmJqaaqJkIiLSMhoLsdzc3DLHGxkZKVxLRkRE9CreAJiIiHQWQ4yIiHQWQ4yIiHSWSiH27NkzhISE4MSJE1VdDxERkcpUCrFatWrhxx9/RGpqalXXQ0REpDKVdye2bt2ad8ogIiKtonKIzZkzB5GRkTh8+HBV1kNERKQyla8TW716NYRCIUaOHAkrKys0adIERkZGcm0EAgGioqLUXiQREZEyKofY9evXIRAI0KhRIwBASkqKQhtlNwgmIiKqKiqH2D///FOVdRAREVUYrxMjIiKdVaEQKy4uRlRUFD799FP4+vri33//BfDyPoh79uzBgwcPqmCGPAwAACAASURBVKRIIiIiZVQOsby8PPTp0wdTpkzB3r17cfToUWRlZQEATE1N8c0332DdunVVVigREdHrVA6x77//HtevX8dvv/2GS5cuQSKRyMbp6elh0KBBOHr0aJUUSUREpIzKIXbgwAFMnjwZvXr1UnoWop2dHe7du6fW4oiIiMqicojl5uaiadOmpY6XSCQoKipSS1FERESqUDnEbGxscPXq1VLHnzlzBvb29mopioiISBUqh5iPjw8iIyNx5swZ2TDpbsXw8HDs378fo0aNUn+FREREpVD5YucvvvgCf/75Jzw9PWFvbw+BQIDZs2cjOzsbGRkZ8PDwwJQpU6qyViIiIjkqh1jNmjURFRWF3377Db///jsEAgFevHiBtm3bwtvbG8OHD+dtp4iI6K1SOcSkfHx84OPjU+mOV6xYgX379uHmzZswMDCAs7Mz5s6dC0dHR1kbiUSCxYsXIyIiArm5uWjfvj2WLVuGli1bVrp/IiLSfW9026l///0Xhw4dwqFDh/Dvv//KXTOmqoSEBEyYMAGHDx9GTEwM9PX1MXjwYOTk5MjarFq1CqGhobKnSltYWGDIkCF48uTJm5RNRETVTIW2xHbv3o25c+ciLS1NFlwCgQBWVlaYO3duhbbQoqOj5V6Hh4fDxsYGSUlJ6N+/PyQSCcLCwuDv7w8vLy8AQFhYGBwcHLBr1y588sknFSmdiIiqIZVDbOvWrfj000/h4OCA77//Hvb29pBIJPjf//6HyMhITJkyBUVFRRg9evQbFZKfn4+SkhIIhUIAwN27d5GRkYEePXrI2hgZGaFLly44d+4cQ4yIiFQPsRUrVqB9+/bYv38/atWqJTdu0qRJGDBgAFasWPHGITZ79my0bt0aHTt2BABkZGQAACwsLOTaWVhYID09vdT5JCcnv1H/JE/blmO+uAhisVjTZeBF8Qu11FHZeairDnXIF+dr3fqibfVUR29rGTs4OJQ5XuUQu3//PiZPnqwQYABQq1Yt+Pr6Yt68eRUuEAC+/vprJCUlITY2Fnp6enLjXj/jUSKRlHkWZHlvmMqXnJysdcvxyo27MDY21nQZ0NfTr3QdYrG40vNQRx3qYmJsAgcHW02XIaON6291o03LWOUTO1q0aFHmFlBaWhqaN29e4QKCgoKwe/duxMTEoEmTJrLhIpEIAPDw4UO59o8ePVLYOiMioneTyiE2f/58REREYM+ePQrjdu/ejcjISCxYsKBCnQcGBmLXrl2IiYnBe++9JzfO1tYWIpEIcXFxsmHPnj3D2bNn0alTpwr1Q0RE1ZPKuxPXrFmDevXqYcKECZg9ezaaNm0KgUCAW7duITMzE3Z2dli9ejVWr14tm0YgECAqKkrp/GbNmoWdO3diy5YtEAqFsmNgxsbGMDExgUAggJ+fH5YvXw4HBwfY29tj2bJlMDY2xrBhwyr5tomIqDpQOcSuX78OgUCARo0aAXi5+xAADA0N0ahRIxQWFuK///6Tm6asY1cbNmwAANnp81KBgYEICgoCAHz++ecoKChAQECA7GLn6OhomJqaqlo2ERFVYyqH2D///KPWjnNzc8ttIxAIEBQUJAs1IiKiV73RHTuIiIi0AUOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0lsoh1rZtWxw8eLDU8bGxsWjbtq1aiiIiIlKFyiGWkpJS5l2zxWIx7t27p5aiiIiIVFGh3Yll3YHj5s2bvJMGERG9VWXesWPbtm3Yvn277PWyZcsQERGh0C43NxdXr15F37591V8hERFRKcoMMbFYLLsxLwDk5eWhpKREro1AIEDt2rXx8ccfY/bs2VVTJRERkRJlhtikSZMwadIkAECbNm2wePFiDBgw4K0URkREVB6VbwB8+fLlqqyDiIiowlQOMaknT54gNTUVOTk5kEgkCuNdXFzUUhgREVF5VA6xnJwcBAYGYs+ePSguLlYYL5FIIBAIkJ2drdYCiYiISqNyiH3xxRfYv38/Jk2aBBcXFwiFwqqsi4iIqFwqh9ixY8cwZcoULFq0qCrrISIiUpnKFzsbGBjAzs6uKmshIiKqEJVDzMvLC0ePHq3KWoiIiCpE5RD77LPP8ODBA0ydOhXnz5/HgwcPkJmZqfCPiIjobVH5mFj79u0hEAhw6dIlREVFldquImcnnjlzBmvWrMHff/+N9PR0hIaGYvTo0bLxfn5+cre9AgBnZ2ccO3ZM5T6IiKj6UjnEvvrqqzJvAPwmxGIxHB0dMXLkSEydOlVpm+7duyM8PFz22sDAQK01EBGR7lI5xIKCgtTeeZ8+fdCnTx8AwLRp05S2MTQ0hEgkUnvfRESk+97oyc7FxcXIzs7Gixcv1F2PgrNnz8Le3h7t27fHjBkzeNyNiIhkKnTbqb/++gvz58/H2bNn8fz5c+zZswfdunVDVlYW/Pz8MH36dHTr1k1txfXq1QuDBg2Cra0tUlJSsHDhQnh6euLkyZMwNDRUOk1ycrLa+n+XadtyzBcXlflQ1rflRfELtdRR2Xmoqw51yBfna936om31VEdvaxk7ODiUOV7lEPvjjz/g6ekJkUiEESNGIDIyUjauXr16yM/Px6+//qrWEBs6dKjs/52cnNCuXTu0bt0ahw8fhqenp9JpynvDVL7k5GStW45XbtyFsbGxpsuAvp5+pesQi8WVnoc66lAXUxMTFCneRvWtq29WByILM61cf6sbbVrGKofYggULYGdnh+PHj0MsFsuFGAC4urpi586dai/wVQ0bNoSVlRVu3bpVpf0QkerynjzFyk17NF0G5swYDZGFmabLoLdM5RD766+/8O2336JWrVp4+vSpwnhra2u5B2hWhaysLKSnp/NEj7coIzMHj3Iea7oMFBYVaboEItJCKodYjRo1UKNG6eeBZGRkwMjIqEKd5+fny7aqSkpKkJqaisuXL8PMzAxmZmZYvHixbBdmSkoK5s+fDwsLCwwcOLBC/dCbe5TzGPNXb9V0GfAfP0TTJRCRFlL57MR27dohNjZW6biioiL89ttv6NixY4U6v3jxItzc3ODm5oaCggIEBwfDzc0NP/zwA/T09HD16lWMGjUKzs7O8PPzg729PY4cOQJTU9MK9UNERNWTyltiM2fOxLBhw/Dpp5/Cx8cHAPDgwQMcO3YMy5Ytw+3btxEaGlqhzl1dXZGbm1vq+Ojo6ArNj4iI3i0qh5i7uzvCw8MREBCAbdu2AXh5WyiJRIK6detiw4YN6NChQ5UVSkRE9LoKXSc2bNgwDBgwAHFxcfjf//6HkpISNG3aFD179oSJiUlV1UhERKRUhUIMAGrXrg0PD4+qqIWIiKhCVD6x4+DBgwgICCh1fEBAQKknfhAREVUFlUNszZo1Sq8Pk3r27BlWrVqllqKIiIhUoXKIXb16Fe3atSt1fNu2bXH9+nW1FEVERKQKlUPsxYsXKCgoKHV8QUEBCgsL1VIUERGRKlQOMUdHR8TExKCkpERhXElJCWJiYtCiRQu1FkdERFQWlUNs6tSpuHDhAkaOHIlLly6hsLAQhYWFuHTpEkaNGoULFy5gypQpVVkrERGRHJVPsR86dChu376N4OBgHD16FAAgEAggkUggEAgQGBgIX1/fKiuUiIjodRW6TmzWrFkYNmwY9u3bhzt37kAikaBp06YYNGgQmjRpUkUlEhERKadSiBUUFGD48OHw9fXFmDFj8Nlnn1V1XUREROVS6ZiYkZER/v77bxQXF1d1PURERCpT+cSOrl27IjExsSprISIiqhCVQywkJAR//fUXvvvuO9y5c0fpqfZERERvk8ondnTo0AESiQShoaEIDQ1FjRo1ULNmTbk2AoEAaWlpai+SiIhIGZVDbMiQIRAIBFVZCxERUYWoHGJhYWFVWQcREVGFqXxMjIiISNtUKMRSUlIwY8YMtGvXDo0bN0ZCQgIAICsrC19++SUuXbpUJUUSEREpo3KI/ffff+jWrRv27t0LOzs7iMVi2XVj9erVw/nz57Fhw4YKdX7mzBmMGDECLVu2hFAoxNatW+XGSyQSBAcHo0WLFmjQoAE8PDxw7dq1CvVBRETVl8ohNnfuXJiamuL8+fNYt24dJBKJ3Pg+ffogKSmpQp2LxWI4Ojpi8eLFMDIyUhi/atUqhIaGIiQkBCdOnICFhQWGDBmCJ0+eVKgfIiKqnlQOscTEREycOBGWlpZKz1Js3Lgx0tPTK9R5nz59MGfOHHh5eaFGDflSJBIJwsLC4O/vDy8vLzg6OiIsLAz5+fnYtWtXhfohIqLqqUIPxTQ2Ni51fE5ODvT09NRSFADcvXsXGRkZ6NGjh2yYkZERunTpgnPnzqmtHyIi0l0qn2Lv6OiI+Ph4TJgwQWGcRCLBvn370K5dO7UVlpGRAQCwsLCQG25hYVHmFl9ycrLaaniXSZdjvrgIYrFYw9UAL4pfVKs6KjsPbVkegPbUki/Ol623/B6oem9rGTs4OJQ5XuUQ8/Pzw8SJE7FkyRJ4e3sDePlE5xs3biA4OBgXL17Ezp07K1etEq/vupQ+v6w05b1hKl9ycrJsOV65cbfMLfC3RV9Pv9rUIRaLKz0PbVkegPbUYmJsAgcHW7n1l6qGNi3jCj0U8969e1i0aBEWL14sGwYAenp6WLhwIXr37q22wkQiEQDg4cOHaNSokWz4o0ePFLbOiIjo3VShh2L6+/tj2LBhiImJwa1bt1BSUoKmTZvC09MTtra2ai3M1tYWIpEIcXFx+OCDDwAAz549w9mzZzF//ny19kVERLqp3BArLCzEwYMHcefOHZibm6Nv376YNm2aWjrPz8/HrVu3ALzcNZmamorLly/DzMwMjRs3hp+fH5YvXw4HBwfY29tj2bJlMDY2xrBhw9TSPxER6bYyQywjIwMDBgzA7du3ZdeFGRsbY+fOnXBxcal05xcvXsSgQYNkr4ODgxEcHIyRI0ciLCwMn3/+OQoKChAQEIDc3Fy0b98e0dHRMDU1rXTfRESk+8oMsYULF+LOnTuYNm0a3NzccOvWLSxduhRfffUVzpw5U+nOXV1dkZubW+p4gUCAoKAgBAUFVbovIiKqfsoMsRMnTmDkyJFYuHChbJilpSUmTpyI+/fvw9rausoLJCIiKk2ZFztnZGSgU6dOcsM6d+4MiUSC1NTUKi2MiIioPGWGWHFxMWrVqiU3TPr62bNnVVcVERGRCso9O/HOnTu4cOGC7PXjx48BvLzYzcTERKF9+/bt1VgeERFR6coNMekZg6/76quv5F5L76SRnZ2tvuqIiIjKUGaIhYaGvq06iIiIKqzMEBs1atTbqoOIiKjCVH4UCxERkbZhiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc5iiBERkc7S6hALDg6GUCiU+/fee+9puiwiItIS5T4UU9McHBywf/9+2Ws9PT0NVkNERNpE60NMX18fIpFI02UQEZEW0urdiQBw584dtGzZEm3atMH48eNx584dTZdERERaQqu3xJydnbF27Vo4ODjg0aNHWLp0Kfr06YOkpCSYm5srnSY5OfktV1k9SZdjvrgIYrFYw9UAL4pfVKs6KjsPbVkegPbUki/Ol623/B6oem9rGTs4OJQ5XqtDrHfv3nKvnZ2d0a5dO2zbtg2ffvqp0mnKe8NUvuTkZNlyvHLjLoyNjTVcEaCvp19t6hCLxZWeh7YsD0B7ajExNoGDg63c+ktVQ5uWsdbvTnyViYkJWrRogVu3bmm6FCIi0gI6FWLPnj1DcnIyT/QgIiIAWr478dtvv0W/fv3QqFEj2TGxp0+fYuTIkZourcplZObgUc5jjfSdLy7ClRt3AQCFRUUaqYGISBVaHWJpaWmYOHEisrKyUL9+fTg7O+Po0aOwsbHRdGlV7lHOY8xfvVUjfb96zMZ//BCN1EBEpAqtDrFNmzZpugQiItJiOnVMjIiI6FVavSVGRKQqgeDlJSGvHtPVhPpmdSCyMNNY/+8ahhgRVQt5T55i5aY9arkOrzLmzBjNEHuLuDuRiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0FkOMiIh0lk6E2IYNG9CmTRuIRCJ069YNiYmJmi6JiIi0gL6mCyhPdHQ0Zs+ejeXLl6Nz587YsGEDfHx8kJSUhMaNG6u9v4zMHDzKeaz2+VZUYVGRpksgojcgEABXbtzVdBmob1YHIgszTZdR5bQ+xEJDQzFq1Ch8/PHHAIClS5fi+PHj2LRpE+bOnav2/h7lPMb81VvVPt+K8h8/RNMlENEbyHvyFCs37dF0GZgzY/Q7EWKC3NxciaaLKE1RUREaNmyIjRs3YvDgwbLhs2bNwtWrV3Hw4EENVkdERJqm1cfEsrKyUFxcDAsLC7nhFhYWePjwoYaqIiIibaHVISYlEAjkXkskEoVhRET07tHqEKtXrx709PQUtroePXqksHVGRETvHq0OMQMDA7Rr1w5xcXFyw+Pi4tCpUycNVUVERNpC689OnD59OqZMmYL27dujU6dO2LRpEx48eIBPPvlE06UREZGGafWWGAB4e3sjODgYS5cuhaurK5KSkhAVFQUbGxtNl1at5eTkICAgAB06dECDBg3g5OSEmTNnIjs7W9Ol6SxetF91VqxYAXd3dzRu3Bh2dnbw9fXF1atXNV1WtbV8+XIIhUIEBARouhTtDzEAmDhxIv755x88fPgQp06dgouLi6ZLqvbS09ORnp6O77//HomJiQgPD0diYiImTJig6dJ0kvSi/S+//BKnT59Gx44d4ePjg3v37mm6tGohISEBEyZMwOHDhxETEwN9fX0MHjwYOTk5mi6t2jl//jwiIiLg5OSk6VIAaPl1YqRdjhw5Al9fX9y9exd16tTRdDk6pWfPnnBycsLq1atlwz744AN4eXlVyUX777r8/HzY2Nhg69at6N+/v6bLqTby8vLQrVs3rFq1CkuWLIGjoyOWLl2q0Zp0YkuMtMOTJ09gaGiI2rVra7oUnVJUVIRLly6hR48ecsN79OiBc+fOaaiq6i0/Px8lJSUQCoWaLqVa8ff3h5eXF7p166bpUmS0/sQO0g65ublYtGgRxo4dC319rjYVwYv2377Zs2ejdevW6Nixo6ZLqTYiIiJw69YthIeHa7oUOdwSe8csXLgQQqGwzH/x8fFy04jFYowcORINGzbE/PnzNVS57uNF+2/H119/jaSkJPz666/Q09PTdDnVQnJyMubPn4/169fDwMBA0+XI4U/qd4yfnx+GDx9eZptGjRrJ/j8/Px8+Pj4AgJ07d6JWrVpVWl91xIv2356goCBER0dj3759aNKkiabLqTb++OMPZGVl4cMPP5QNKy4uRmJiIjZt2oS0tDQYGhpqpDaG2DumXr16qFevnkptnzx5Ah8fH0gkEuzatQsmJiZVXF319OpF+6/eyDouLg6enp4arKx6CQwMRHR0NPbv34/33ntP0+VUKx4eHnj//fflhk2fPh12dnaYOXOmRrfOGGKk1JMnT+Dt7Y0nT55g69atePr0KZ4+fQoAMDMz07pdCtqOF+1XrVmzZmHnzp3YsmULhEIhMjIyAADGxsb88aUG0kMNr6pduzbMzMzg6OiooapeYoiRUpcuXcL58+cBAO3bt5cbt2/fPri6umqiLJ3l7e2N7OxsLF26FBkZGWjZsiUv2lejDRs2AAC8vLzkhgcGBiIoKEgTJdFbwuvEiIhIZ/HsRCIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMSIi0lkMMS23detWufsaWllZoXXr1hg9ejT27NmDkpISufZ3796FUCjE1q1b5YYvX74crVq1Qr169dC1a1cAQEZGBkaMGIEmTZpAKBRi7dq1b+19aQvp8nr1X7NmzTBgwAAcP378jeYZHx+P4OBghc+munj+/Dk2btyI/v37w9bWFvXr10fz5s3h6+uLHTt24MWLF5ouscL8/PwgFArh5OSk9HMLDg6WrR+6+P6qM17srCMiIiJgZWWFwsJCpKam4siRI5gwYQJ++eUX7NixA0ZGRgCABg0a4OjRo2jatKls2gsXLmDBggWYMWMGPDw8ZHcwWLJkCRITExEaGooGDRq80xfezpw5U/bcqYcPH2L9+vXw9fXFoUOH0KFDhwrNKyEhASEhIQgICECNGtXrd6L0VmSXLl3C2LFj8dlnn6Fu3bpIS0vDoUOHMH36dBgYGMDb21vTpVZY7dq1kZ6ejvj4eIVHjURFRcHU1BRPnjzRUHVUGoaYjmjdujWaNWsmez1ixAh4eXlh3LhxmDNnjuzBdIaGhgpfuv/99x8AYPz48XI3Rf3vv//g5OSEQYMGqaXGwsJCjd0EtLKaNGkit9zc3d3RpEkT7N+/v8IhVp199dVXuHjxIg4cOABnZ2e5cT4+Pvj777/x7NmzMuehreuJUCiEg4MDduzYIRdiZ8+exZ07dzBixAhs375dgxWSMtXrZ+I7xsvLCwMGDEBkZKTsvoav70708PDAtGnTAADt2rWDUCiU7TpJSEjA2bNnZbtJ7t69CwC4c+cOJk2aBDs7O1haWqJr167Yt2+fXN/S3StXr16Ft7c3rK2tMW7cOADA06dPMXfuXLRp0wYWFhZo06YNli1bJrebJj4+HkKhEAcPHkRAQACaNWsGOzs7TJ48Gbm5uXJ9vXjxAitXrkSnTp0gEolgZ2eHoUOH4saNG7I2WVlZmDlzJlq2bAlLS0t06NABv/zyyxsv21q1aqFmzZp4/vy53PDy+gkODkZISAgAoH79+nL3nPvwww/x2Wefydrm5eXB3Nxc4d5zffv2lS1L6ftfsWIFOnToAEtLS7Ro0QLffPONQlioe7m/Li0tDVFRURg3bpxCgEm1bdsWnTp1klsepa0nDx48wJQpU9CsWTNYWlqiS5cu2Llzp9z8pNO/zs/PD61bt5a9lq73GzZswNdffw17e3s0bNhQ9iRyVY0YMQL79u2T/T0BwI4dO/Dhhx+WuqciIiICLi4uEIlEaNasGT799FPk5OTItVm3bh169+6NJk2awMbGBr169cLhw4fl2kjfw+bNm7Fo0SI0b94cNjY28PX1xf379+Xa/vbbb3B1dYW1tTVsbGzQpUsXbN68WeX3WZ1wS0zH9enTBwcOHMDFixfh4uKiMH758uWIiorCihUr8Ouvv6JBgwYQiUQYP348/P39oaenh+XLlwN4uSsyNTUVvXr1goWFBX744QfUr18f0dHRGDt2LLZu3YoBAwbIzX/UqFH46KOP8Pnnn6NGjRp48eIFhg4diuvXryMgIABOTk44f/48li5dipycHCxatEhu+tmzZ6Nv377YsGEDkpOTMXfuXNSoUQM///yzrM348eNx4MAB+Pn5oXv37nj27BkSExPx4MEDvPfee3j8+DH69u2LZ8+eYfbs2bC1tcXx48cxc+ZMFBYWYsqUKeUux5KSEtmxjszMTKxevRrPnj2Tu8u8Kv2MHTsWaWlp+PXXXxEbGyv3PCtXV1e5L66EhAQYGhoiLS0NN2/ehL29PcRiMf766y8sXrxY1m7y5MmIjY3F559/jk6dOuG///7DokWLkJKSgl9//RUAqmS5vy4hIQHFxcXo169fucvzda+vJ2KxGB4eHsjNzcWcOXNgbW2NqKgoTJkyBQUFBXIhXhE//vgjWrVqhdDQUGRmZmLBggXw9vZGUlISatasWe70np6emDVrFg4cOAAfHx8UFhbi999/x4IFCxSCBADmzZuHn376CVOmTMGCBQuQlpaGRYsW4dq1azhy5Ijs809JScFHH30EW1tbvHjxArGxsfD19cVvv/2G3r17y81zxYoV6NSpE3766SdkZmbi22+/xaRJk3Dw4EEAL7cMJ0+eLOuzpKQEN27cQF5e3hstM13HENNx0md/Se/a/boWLVrIdiG2adMGtra2AAAbGxuYmppCT09PbnfZ4sWLIZFIcODAAZibmwMAevbsifv37+OHH35QCLEpU6bAz89P9nrHjh04e/YsDhw4IAtV6a6ZkJAQ+Pv7yz1Dq0uXLrJdoT169MDNmzcRGRmJsLAwCAQCnDp1CjExMVi8eDGmTp0qm27gwIGy///5559x7949JCYmws7ODgDQvXt35OXlISQkBBMmTCj3adT+/v7w9/eXvTY0NMTKlSvRuXPnCvVjbW0NKysrAICzs7Ncv66urli3bh1SUlJgY2MjO/Zy48YNxMfHw97eHmfPnsXz589lN1hOTExEdHQ0wsLCMHLkSFmfZmZmmDx5Mi5fvow2bdpg165dal3uyqSlpQGQf94c8PLhnsXFxbLXNWrUUDgW+Pp6sm7dOvzvf/+Tu5l079698fDhQyxcuBAfffTRGz3Q0sTEBNu3b5f1b29vj379+mH79u0YO3ZsudMbGxtj4MCB2LFjB3x8fHDw4EEUFRVh8ODBCA0NlWt79+5drF69GoGBgQgMDJQNl/Z56NAh2Xq6cOFC2fiSkhJ069YNN2/exKZNmxRCrHHjxrIbGgMvt/6/++47pKeno2HDhvjzzz9Rt25duR86PXr0qMBSql64O1HHSSQv79+sricEHz9+HL1790adOnXw4sUL2b+ePXvi33//xePHj+Xavxom0ukbN26MTp06yU3fo0cPPH/+XHZnfKm+ffvKvXZ0dERhYaHsAZJxcXEQCAT4+OOPy6y5ffv2sl+5r9acnZ2N69evl/u+Z82ahbi4OMTFxcm2PP39/bF792619dO1a1fUqFEDp0+fBgCcPn0abm5ucHNzkxvWoEED2fOwjh8/DgMDA3h6eiosT+BlyFXFcldGuq69buXKlahfv77s36s/NqReX08SExNhZWWl8DSEyVOPUAAACCZJREFU4cOH49GjRyp9Zsp4enrKBWjnzp1hbW2t8P7LMnLkSJw8eRIZGRnYsWMHBgwYgDp16ii0O3nyJEpKSjB8+HC5Ze7s7Iw6derIPhvg5VMhfH194eDggHr16qF+/fqIi4vDzZs3Fear7LMBgNTUVADA+++/j9zcXNkWenm7gas7bonpOOkuDpFIpJb5ZWZmYseOHdixY4fS8dnZ2XJ/0A0aNFCY/t69e6hfv36p07/KzMxM7rX0OWXS4z3Z2dkwMzOTnX1ZWs23bt1SuU9lbGxs5B7616NHD9y9exdBQUHw9vaGQCCodD9mZmZo1aoV4uPj0b9/f1y7dg2urq4QiUSYPXs2gJfHrF79Ys/MzERRURGsra3L7FPdy10ZaQ2pqalwcHCQDR81ahS6d+8OALKtxde9vp7k5OQoXWelw14/pqQqS0tLhWEWFhZIT09XeR5ubm5o0KAB1q5di+PHj5d6MkdmZiYAKDwsUkq6zFNTU+Hp6YkWLVpgyZIlaNSoEfT19bFo0SLZSVevKu+z6dq1KyIiIhAeHo4xY8YAAFxcXLBo0SK0atVK5fdZXTDEdNzhw4dRq1YttGvXTi3zMzc3x4cffii3a+1VDRs2lHv9+hagubk5bG1tSz2poqKn8derVw85OTkoKCgoNcjMzc1hYWEht3vlVfb29hXqU6pFixY4cuQIMjMzYWlpqZZ+XF1dsWfPHsTHx8Pc3BytWrVCgwYNkJmZiaSkJFy+fFnuQZnm5uaoVasWDh06pHR+0nBQ93JXRrolGRsbC3d3d9lwkUgkC5/Sjju9vp6YmZkp3QqR7haX7squVasWAKCoqEjuQayl/WBQtiWZmZkpdxJIeWrUqAEfHx+sXr0aFhYWpe6qk9a4Z88epSefSMPo+PHjePz4MTZv3iz3Y+TVk0cqysvLC15eXsjPz0dCQgLmzZuHYcOG4erVq9Xuso7yMMR0WExMDA4dOoSpU6eidu3aaplnz549cf78ebRo0aLMrZ+ypo+JiYGxsbFaHhHv7u6OH3/8EZGRkaWeoNGzZ0+sW7cOjRo1kjvuU1lXrlyBgYGBbMtT1X6kp48XFBTA1NRUbpyrqytCQ0OxefNmdO3aFQKBABYWFmjZsiWCg4NRXFwMNzc3ufe2cuVKPH78WOHapVepe7krY2VlBR8fH/zyyy/w8fEp9QxFVbi4uOD3339HUlKS3HHHXbt2wcLCAs2bNwfw8vgQAFy9elX2Qy03Nxd//PGH0ic2x8TEICgoSPZFnpSUhPv371f4MokxY8bgxo0bcHd3L/XYnLu7O2rUqIF79+7JhfrrpGH1asDfvHkT586dkx0/fVMmJibo168f7ty5g9mzZyM7O7vUrfHqiiGmI/755x9kZWWhqKgIqampOHz4MH7//Xe4u7tj7ty5auvn66+/Rs+ePTFgwABMmjQJNjY2yM3NxbVr13Dnzh2Fg9uvGz58OLZu3QovLy9Mnz4drVu3RlFREW7fvo1Dhw5h69atFQpcNzc3eHp64ptvvsH9+/fh5uaG58+fIzExEX369IGrqyumTZuGPXv2oH///pg2bRrs7e3x9OlTJCcnIzExUaVre+7cuSM7bpKbm4vY2FgcP34cEydOlG0NqNqP9Av4p59+Qu/evaGnpyfb5dSlSxfo6enh1KlTWLZsmaz/rl27Yv369WjUqJHctXyurq7/187dvKTSxmEc/3YoioRReoGwgiQLXIRS7UIrimgTLQoiQUFoERVUGyGj6IXey6QWNWiCEEkR7dpIkIto138QQhEVJdhChFY9i8MRek49Rw49C8/5fdY39zCzmGvue64Zent7cTqdDA8P09DQwLdv37i9vSUSiTA7O4vRaPzy6/6ZtbU1YrEYXV1dOJ1OWlpa0Ol0JBIJLi8veXp6+jBc/s1ut7O7u4vD4WBqagq9Xs/R0RHn5+f4fL50cLS3t6MoCqOjo0xMTPD6+srW1hYajebDeZPJJHa7HZfLRTweZ25ujurq6k+3OT9jNBo5ODj4zzEGg4GxsTHcbjfX19c0NTVRUFDA3d0d0WgUh8OBzWajpaWF3NxcBgcHGRkZ4fHxkaWlJSoqKn7rry4LCws8Pz9jtVopKyvj/v4eVVWpq6v76wIMJMSyxo9iQ0FBASUlJZjNZoLBIN3d3V9W6oDvT77n5+csLy8zPz9PPB6nqKgIk8mU0Y0gLy+Pk5MTNjc3CYVC3NzcUFhYiMFgoKOj492WUKaCwSA+n49wOMzOzg6KolBfX59um2m1WiKRCCsrK/h8Ph4eHtBqtdTU1GT8IbfX68Xr9QKgKApVVVWsr6+/q3pnepzOzk4GBgbY29tjdXWVt7e39Mt3RVGwWCxcXV29W3HZbDb8fv9PRQf43uRTVZX9/X02NjbIz8+nsrKStra29Irw/7juH1EUhdPTU0KhEMfHx4TDYVKpFMXFxVgsFra3t+np6fnlPBqNhtPTU6anp5mZmSGZTGI0GlFVlb6+vvQ4nU7H4eEhHo8Hl8uFXq/H7XYTjUa5uLj4ad7x8XFisRhDQ0OkUimsViurq6sZ1et/x/T0NLW1tQQCAQKBADk5OZSXl9Pc3JxusJpMJvx+P4uLi/T392MwGJiZmeHs7OzDc/iVxsZGVFXF4/GQSCQoLS2ltbWVycnJrz69rJDz8vLyceVICCGyxM3NDWazma2trYyq9OLP8Xe9ARRCCPFHkRATQgiRtWQ7UQghRNaSlZgQQoisJSEmhBAia0mICSGEyFoSYkIIIbKWhJgQQoisJSEmhBAia/0Dg+PX+nc4Ku4AAAAASUVORK5CYII=\n",
"text/plain": [
"

Maternal Smoker | Birth Weight average |
---|---|

False | 123.085 |

True | 113.819 |

Maternal Smoker | Birth Weight |
---|---|

False | 120 |

False | 113 |

True | 128 |

True | 108 |

False | 136 |

False | 138 |

False | 132 |

False | 120 |

True | 143 |

False | 140 |

... (1164 rows omitted)

" ], "text/plain": [ "Maternal Smoker | Birth Weight\n", "False | 120\n", "False | 113\n", "True | 128\n", "True | 108\n", "False | 136\n", "False | 138\n", "False | 132\n", "False | 120\n", "True | 143\n", "False | 140\n", "... (1164 rows omitted)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "smoking_and_birthweight" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are 1,174 rows in the table. To shuffle all the labels, we will draw a random sample of 1,174 rows without replacement. Then the sample will include all the rows of the table, in random order. \n", "\n", "We can use the Table method `sample` with the optional `with_replacement=False` argument. We don't have to specify a sample size, because by default, `sample` draws as many times as there are rows in the table." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "shuffled_labels = smoking_and_birthweight.sample(with_replacement = False).column(0)\n", "original_and_shuffled = smoking_and_birthweight.with_column('Shuffled Label', shuffled_labels)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "Maternal Smoker | Birth Weight | Shuffled Label |
---|---|---|

False | 120 | True |

False | 113 | False |

True | 128 | False |

True | 108 | True |

False | 136 | False |

False | 138 | False |

False | 132 | True |

False | 120 | False |

True | 143 | True |

False | 140 | False |

... (1164 rows omitted)

" ], "text/plain": [ "Maternal Smoker | Birth Weight | Shuffled Label\n", "False | 120 | True\n", "False | 113 | False\n", "True | 128 | False\n", "True | 108 | True\n", "False | 136 | False\n", "False | 138 | False\n", "False | 132 | True\n", "False | 120 | False\n", "True | 143 | True\n", "False | 140 | False\n", "... (1164 rows omitted)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "original_and_shuffled" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each baby's mother now has a random smoker/non-smoker label in the column `Shuffled Label`, while her original label is in `Maternal Smoker`. If the null hypothesis is true, all the random re-arrangements of the labels should be equally likely.\n", "\n", "Let's see how different the average weights are in the two randomly labeled groups." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "Shuffled Label | Birth Weight average |
---|---|

False | 119.277 |

True | 119.752 |