{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 07 - Beyond Confounders\n",
    "\n",
    "## Good Controls\n",
    "\n",
    "회귀 모델에 통제변수를 추가하는 것이 인과효과를 식별하는데 어떻게 도움이 되는지 알아봤습니다. 통제변수가 교란 요인(confounder)인 경우, 모형에 추가하는 것이 좋을 뿐만 아니라 필수 사항입니다. 무의식적으로 이것을 생각했을 때 자연스런 반응은 측정할 수 있는 모든 것을 모형에 집어넣는 것입니다. 오늘날의 빅데이터 세계에서는 1000개 이상의 변수가 될 수 있습니다. 하지만 이렇게 많은 변수는 불필요할 뿐만 아니라 인과효과를 식별하는데 도움이 되지 않을 수 있습니다. 이제 교란 요인이 아닌 통제변수에 주의를 돌리겠습니다. 먼저, 통제변수의 좋은 점들을 살펴보겠습니다. 그 후, 인과효과를 식별하는데 해로운 통제변수에 대해 알아보겠습니다.\n",
    "\n",
    "동기 부여를 위한 예시로 우리가 핀테크 데이터 수집팀의 데이터사이언티스트라고 가정해보겠습니다. 과업은 사람들에게 빛를 갚을 것을 요청하는 이메일을 보내는 행위의 효과를 파악하는 것입니다. 반응변수는 연체고객으로부터의 지불금액(payments)입니다.\n",
    "\n",
    "이 질문에 답하기 위해, 데이터 수집팀은 연체고객 중 랜덤하게 5000명을 선택하여 랜덤 테스트를 실시합니다. 그리고 고객의 수만큼 동전을 던집니다. 동전이 앞면일 경우 고객이 이메일을 수신하고, 뒷면이 나올 경우 이메일을 수신하지 않은 통제 그룹으로 남습니다. 이 테스트를 통해 이메일이 얼마나 많은 추가 비용을 발생시키는지 알 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from scipy import stats\n",
    "from matplotlib import style\n",
    "import seaborn as sns\n",
    "from matplotlib import pyplot as plt\n",
    "import statsmodels.formula.api as smf\n",
    "import graphviz as gr\n",
    "\n",
    "style.use(\"fivethirtyeight\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>payments</th>\n",
       "      <th>email</th>\n",
       "      <th>opened</th>\n",
       "      <th>agreement</th>\n",
       "      <th>credit_limit</th>\n",
       "      <th>risk_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>740</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2348.495260</td>\n",
       "      <td>0.666752</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>580</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>334.111969</td>\n",
       "      <td>0.207395</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>600</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1360.660722</td>\n",
       "      <td>0.550479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>770</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1531.828576</td>\n",
       "      <td>0.560488</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>660</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>979.855647</td>\n",
       "      <td>0.455140</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   payments  email  opened  agreement  credit_limit  risk_score\n",
       "0       740      1     1.0        0.0   2348.495260    0.666752\n",
       "1       580      1     1.0        1.0    334.111969    0.207395\n",
       "2       600      1     1.0        1.0   1360.660722    0.550479\n",
       "3       770      0     0.0        0.0   1531.828576    0.560488\n",
       "4       660      0     0.0        0.0    979.855647    0.455140"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv(\"./data/collections_email.csv\")\n",
    "data.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터가 랜덤이기 때문에 단순한 평균 차이가 ATE를 추정한다는 것을 알고 있습니다. 즉, 무작위화 외에는 어떤 것도 treatment의 원인이 될 수 없으므로 잠재적 결과는 treatment와 독립입니다: $(Y_0, Y_1)\\perp T$. \n",
    "\n",
    "$\n",
    "ATE = E[Y|T=1] - E[Y|T=0]\n",
    "$\n",
    "\n",
    "추정량에 대한 신뢰 구간을 계산하기 위해 선형회귀모형을 이용할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in means: -0.6202804021329484\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>  669.9764</td> <td>    2.061</td> <td>  325.116</td> <td> 0.000</td> <td>  665.937</td> <td>  674.016</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>email</th>     <td>   -0.6203</td> <td>    2.941</td> <td>   -0.211</td> <td> 0.833</td> <td>   -6.387</td> <td>    5.146</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Difference in means:\",\n",
    "      data.query(\"email==1\")[\"payments\"].mean() - data.query(\"email==0\")[\"payments\"].mean())\n",
    "\n",
    "model = smf.ols('payments ~ email', data=data).fit()\n",
    "model.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "안타깝게도 ATE 추정치는 -0.62로 꽤 이상합니다. 이메일을 보내는 것이 연체 고객의 평균 지불 금액을 감소시키는 방법일까요? P-value가 너무 높기 때문에 아무런 의미가 없을 수도 있습니다. 이제 어떻게 해야 할까요? 테스트 결과가 명확하지 않고 더 많은 데이터가 필요하다고 말해야 할까요? 아직 이릅니다. \n",
    "\n",
    "데이터에 다른 흥미로운 변수가 있습니다. 예를 들면, `credit_limit`는 연체 전 고객의 신용한도를 나타냅니다. `risk_score`는 이메일을 보내기 전 고객의 예상 리스크입니다. 신용한도와 리스크는 연체고객의 지불금액을 예측하는데 매우 좋은 변수로 보입니다. 하지만 이 변수들이 어떻게 유용할까요?\n",
    "\n",
    "첫째, treatment에 대한 통계적 유의성이 없는 이유를 이해해봅시다. 이 경우처럼 treatment가 outcome에 거의 영향을 미치지 않을 수 있습니다. 생각해보면 사람들이 빚을 갚게 하는 것은 대체로 통제변수 이외의 요인일 것입니다. 사람들은 새로운 직장을 구하고, 재정, 수입 등을 관리하는 것을 통해 빚을 갚습니다. 통계적인 관점에서 **지불금액의 변동성은, 메일을 수신하는 것 이외의 다른 요인에 의해서도 설명되고 있다고 말할 수 있습니다.**\n",
    "\n",
    "이를 시각적으로 이해하기 위해 treatment인 이메일에 대한 연체 고객의 지불 금액(payments)을 표시할 수 있습니다. 회귀모형의 적합값은 빨간색으로 표시했습니다. 더 나은 시각화를 위해 이메일 변수에 약간의 노이즈를 추가하여 0 또는 1에서 값이 겹치지 않도록 했습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcIAAAEfCAYAAADSuMa0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAButklEQVR4nO3dd3xUVd748c8t02eSSSck9N6b0kRdxe7juvay+3NFERawgJ1V176IBXVdxAI+rqvuqqirrj62tSNFUQid0AmE1JlkZjLt3nt+f0wSAUGlJuyc9+uVl86dO5NzM5z53tO+RwkGgwJJkiRJSlNqSxdAkiRJklqSDISSJElSWpOBUJIkSUprMhBKkiRJaU0GQkmSJCmtyUAoSZIkpTUZCCVJkqS0JgOhJEmSlNZkIDwESktLW7oIB0xeQ8s70ssP8hpaC3kNP00GQkmSJCmtyUAoSZIkpTUZCCVJkqS0JgOhJEmSlNZkIJQkSZLSmt7SBZAkSZIOPgGURRUqohZ2Tx4CUFq6UK2UDISSJEn/ZQTwYbnJI4trSVgCYcT54wgfpxRqMhjugewalSRJ+i9TFlWagyBAwhQ8sriWsqgMg3siA6EkSdJ/mYqo1RwEmyQsQUXUaqEStW4yEEqSJP2XKXCp2NVdW392VaHAJb/y90T+VSRJkv7LFLsENwzJbg6Gdk3hhiHZFLvEz7wyPcnJMpIkSf9lFOCUQo3eo/OpiFrYYnX0lxNl9kq2CCVJkv4LKUA7l+CobAV3pEoGwZ8gA6EkSZKU1mQglCRJktKaDISSJElSWpOBUJIkSUprMhBKkiRJaU0GQkmSJCmtyUAoSZIkpTUZCCVJkqS0JgOhJEmSlNZkIJQkSZLSmgyEkiRJUlqTgVCSJElKazIQSpIkSWlNbsMkSZKURgRQFlWoiFoUuFSKXSLtd6aQgVCSJClNCODDcpNHFteSsAR2NbVh7ylpvleh7BqVgFQF2RpV+LZWsDWqoGryHkmSjkS71+Wd96QviyrNQRAgYQkeWVxLWTSdw6BsEUrs+S5xQq9cOkNa3yVK0pFmby2+ro03thVRqzkINklYgoqoRTtX+tZ22SKU9niX+NjSYNrfJUrSkWZvLb6wMwuAApeKXd21XttVhQJXeoeC9L56CdjLXaKZukvc3U91u0iS1LJ2r8tZDpUT23vYajjZGlUocgluGJLdHAybWozFrvSuybJrVGq+S9y5Atm1prvEH47JgXZJat12rstZDpWhhR6eXh4k3wGZjhA3DMnm5EKN3qPz5azRncgWoUTxHu4SJw/w/+guUQ60S1LrtnNdHlbo5pnlQbIcKqplNNfXVSFVBsHdyBahhAKcsttdYqJ8Awq+Xc6TA+2S1LrtXJdXBk0K3WHsKsTjqXpbHbd4e0OEDzaHZI/OTmSLUAJSFaidS3BUtkI7l8AyjR+dIwfaJan1a6rLvf0aXv2H+pqwIBC3UBTZo7M7+Q0m/aSdJ8eoCnKgXZKOEDt3k1qqDQuFu0fkkevUyHKkvvqbenTSnewalfZqT5Njbj06m9mj86mUYwyS1KopwMmFGs5heXxaFiFiwMPfVHNOtwxOaOfl061hIknxo0lx6UgGQmmv9jQ55oFvapkzOp+jshXSvfJIUmu3Lapw5/xK1gTjgIopBM8sC3J570xGtvXQP9cue3SQXaPSbpq6Qrfa8tgRE3hsu7b3ZFeKJB05KqIWDYZANMY6TUmt/XVoKsPaOOVEmUYtGgjnzZvHxRdfTK9evfD7/bz00ku7PC+EYNq0afTs2ZM2bdpw5plnsmrVql3OCQaDjBs3jvbt29O+fXvGjRtHMBjc5ZwVK1Zwxhln0KZNG3r16sX06dMRQt4F7a6pK/TK/1Ry2zf13PRlFcPbeprHE0BOjpGkI0mBS8WtKyg7RTunpuDRoZNXlUGwUYt+o0UiEXr37s0DDzyAy+X60fOPP/44M2fOZPr06XzyySfk5eVxzjnnEAqFms8ZO3YsJSUlzJ07l7lz51JSUsL48eObn6+vr+ecc84hPz+fTz75hAceeIAnnniCv/71r4flGo8UAlgZUvmuMsGJ7b10zrRzWkcPMRN+2yuLLIcqJ8dI0hGmyCWYMjibdj47mQ4Vj01hQn8/Q/Jll+jOWnSM8JRTTuGUU04BYOLEibs8J4Rg1qxZTJ48mbPPPhuAWbNm0a1bN+bOncuYMWNYs2YNH3/8Me+//z5Dhw4F4NFHH+X000+ntLSUbt268dprrxGNRpk1axYul4vevXuzdu1annzySa6++moURd4TNbUE711Uxfq6JB6bwk1H5fBsSYBtEZNOmTamDMpmZL5OW6ecHCNJRwIBfFRu8vKqIL/u6AJVZ1gbF338CoUOWY931mr7uDZv3kxFRQUnnnhi8zGXy8XIkSNZuHAhAIsWLcLr9TJs2LDmc4YPH47H49nlnBEjRuzS4hw9ejTl5eVs3rz5MF1N69Y0KcYSAkUBXVW4e0E1J7T3prpUhODZklosIXejkKQjRVO9roiavLsuwMebQzz0TRWmJevx7lrtrNGKigoA8vLydjmel5dHeXk5AJWVleTk5OzSqlMUhdzcXCorK5vPadu27Y/eo+m5jh077vH3l5aWHlD5D/T1h5qq6YSdWdQkwNSdhKIxAPKdOlETYoYg26Ewrq8fENgQbK4OEotVtmi591Vr/xx+zpFefpDX0FK22vIIRaPNj+PxGHFgXUWAWLKq5Qp2AA7kc+jWrdten2u1gbCl/dQf7ec0dcu2VruvDzyzswNhc+BUwQHYDHDrChkOnXsWVpMwBRl2lb7H53NMt8wj5m6ytX8OP+dILz/Ia2hJzqiCz5UkYQni8RgOhxO7qtC1IIt2Ln9LF2+fHcrPodV2jRYUFABQVbXrnUtVVRX5+fkA5OfnU1NTs8sMUCEE1dXVu5yzp/doei4d7b4+8OttEc7p4gMUEhYYQnDXiDymfVNN3Eyd49IVHvtepmOSpCPFnpLpy8lue9ZqA2GHDh0oKCjg008/bT4Wi8WYP39+85jg0KFDCYfDLFq0qPmcRYsWEYlEdjln/vz5xGKx5nM+/fRTCgsL6dChw2G6mtZl9+TZgbjFgu0R7j8mj1uPzuW+EfmsrI7j1lXyXRp5Lo1QwiKSlGsIJelIIEjd8OY4VB47voCHhmcyZ3S+XDe4Fy3aNRoOh9mwYQMAlmVRVlZGSUkJWVlZtGvXjgkTJjBjxgy6detG165defjhh/F4PJx//vkA9OjRg5NOOokpU6bw2GOPATBlyhROPfXU5ib0+eefz/Tp05k4cSI33ngj69at47HHHuPmm29O2xmje9p/sDxq8fX2KG+uD3FuVx9JoC5hNbcIHZqCW//xHoWSJLUue0qNOKGXk2NkOsS9atFA+P3333PWWWc1P542bRrTpk3jkksuYdasWVx33XVEo1FuuukmgsEgQ4YM4Y033sDn+2F7oNmzZ3PzzTdz3nnnAXD66afz4IMPNj+fmZnJm2++yY033sgJJ5yA3+9n0qRJXH311YfvQluZpi6TpoqS5dC4tJefbeEkUwbnkOuEhKVSNDKftcE4/1qf2rLl2kGyW0WSWrumoQ+PTeHEQg+aqhLXbXxcIeiRkcoPvC2qyD0Jd6IEg0H5zXaQHQmD601dJzVxi9J6i9u/rkRRFJwq/GFgNg8sqiHToWLXFK4dmE0wmqRPjpMTC46cKnMkfA4/5UgvP8hraAnf1gru/6aGoYUeXl1bxxmdM5izPEiOU8NjU7hhcDZvrK2jImoeUXsSpuVkGenQatqzLMehMnNJLYqiUBU1Gd3Bx13zq9FUhS0hg9qYxU1fVuJ32Shway1dbEmSfkaBS2VkWw+zlgX5VTsfs5cHiZmCyqhBIG7xyHe1DCpIrauWexKmyECY5pomzji0VD5CRYGYmeoq0dTUMgqvTcVv1+jpkxNlJKm1K3IJembb8TemVLOEwKaC0Vh9o4bAEDtv2CsnwclAmMYEELEUyhvM1KSYxswxzsagmO3Q2NFgEE5arKiJ8XG5KafJSFIrJoB5VSYoKlVRkwZDoDZOClQVEALcNnWXnetlIn0ZCNNaWVThpRUBHhiVz7ldfEwakM3iHRHuHpGL36ESTFjYVYWJA7KYtz0iu1AkqZUriyrsiMLt8yq4oo+fDzeFGNvXj6YotPXo+B0qF3XzYWv85pdrC1NkZpk0VhO36JTl5LZ5laiKgs+uct2gbLyaxZRBOXxXGUMAH28JEzMEbi3VldrOJYOhJLVGFVGLUEKwJWzyWmk9Z3b0YVfggVH5+GwKK2ri6AoMzncwrCBPzhptJANhGrNpGs8sDxI1BAnTwhLw0Lc1nNbRiymSvLS6jriZGl/olGHbqQslve8eJam1KnCpZNgtnLpCZYPJ8yuDCMBrU7h3RD4zS4K4dYV/n1VEH5+FrMspsms0jUWTZvOmuw5NwW1TUBUY1dZNkUdjxnEFdPKlplWriuxCkaTWrsglaO/T+PPIPDLsKjYVMuwq947IIxQ3uKxXJmN6+0maZksXtVWRLcI0VuBSyXWoODSFUFKwLWzg1BS+2t6A16byl++qGdsvm5qowQntPByTI7tQJKm1atp/8JHFtWQ6VB44Jh8U0IDamMGclXVUR03smsKA3HwG+Fv/2sHDRbYI01hThhlNVdgeSQXBy/v4eWtDiCdLApzUwce9C6vpnuXgr99Vs01OlJGkVqspo0zYECyrTTLp0x1M+byCqCG4a2EN2yMGPrtKVuPaYTnx7QeyRZjm+mbp3DU8j/c3R3CoCq+V1lPVkOo20VTIdqpsDSWoiJpyoowktWIVUYuwIYiZkO1QOb2jD0WBHJdGJ59ObdzCa1Px6j+sHZT1OUUGwjS1c2LeK/tn89KqOnx2lepYaq2gQ1MwLXBqKg5VkRNlJKkV23lNcFuPxikdfDxZEiBpCb7e3sDJHby8UlpPWdigrUcj16HK+rwT2TWapnbek/D99XXcNSKXhCXIcWo4NYWJ/bN4f1OI/+nspdCrM2mgnCgjSa1VWVThye9rGNfXzwnFHp5dHkAIQRu3Tlk4yfubw9x2dC6/7ZmJCrI+70a2CNPUznsSbgwZUBbmqRML2RExyHNpLK2Ocnyxl483hxmcl0NnnyoH1iWplaqIWlRETdbWRDmjSybu0hD5LhWBwGNT2VCXZE1tnPc3hbl2YJasz7uRgTBN7b4n4caQwfeVUV5cXc+2iEGqy0TBqSk4dQ1VVYH0zkcoSa1VU33umu3km4oYdYnUumBX43pCu6YQNWFbxODP39Tw3q+LkPX5BzIQpqnd9yRUFIV+uXamj8pjVU2CIp+NzfUJ+uY6cWmpBfep0ChJUmtT5BLcNiyPdXUGSSvBpP5Z/G1VEF1VKfLqTOyfxTvr68l3abh0hS1hk/IodPeptJOZZWQgTFcKcEqhRq/R+SwPGLhtCtsiFnOW13JcsZebv6ogw67h0Oq5sFsGbl0hELdxQoFceyRJrcnO6weDScGOBpNbh2Rz34h8VgbiRA3BrKW1nN01gy3hemyqwvzyKE8tC9Ahw8a9w3M49QjYj/BQkoEwjSmApoDbpuHUFUoDMS7olsmG+iRum0pV1KCNW2fuuhDXDsxmRU2Sbhk67eQguyS1GjtPfNMbt06ripo8t7KOhCUwBexoMJm9PMjlvTNx6ApeTeG3Pf1oKnxflaBflpsiZ/rWaxkI05gAFtcYhBIWCUvh2eVBnLpKOGlxVV8//9kSZl1dEiFgeU2cLpk2auJy7ZEktSY7T3wzLMh1qRT57GwNGwghEEBbr07MEHTJtPP2+jrO7JzBy98HiJsCn02lq9/O+e3Tt1Uol0+ksbKowsqaBBvrDWYurW2uTAlT8NyKIMMLPQiRWlNoCXiyJIBNk7vUS1Jr0jRRBlIbaw8r8LA2EMemgiUgaUFZyMCmKqyqTTAo382zy4OYVmps0LBE2meakS3CNFYRtTAExE1BWdgg361TGzPJd2nYVBiQ58Cu+Tkq30l5OMk5XXw0GHLSjCS1JjtPfIsYArdN4bOyCFf1zWLu2jrO755BV78Dj03BNAWqKvjn2tSMUbumUOjRgPTONCMDYRorcKk4NGgKazVRk0KPRpZDZXR7L39dUkt9wuL9TWF+08XHm+vq+XxrhKlHZ3NKmg+uS1JroQAnF2o4h+Xx5fYo7X02/rYySElVAzcMzqYsYnLzlxXETEGRV2fqUbl0ztDx2lScGthV0j5zlOwaTWPFLsGQPDtdMnUmDchCkBpjGFLg5pU19YSTqbVIW0IGTy8Lcl63DEDIneolqZXZFlW4f2EVH28O8feVAX7f20/3LCcbQwZPLA0QMVKTZspCBtO/rea6wTlk2pXmIJjuW6zJFmEaU4DjCzS+D2oUem38qshFIC6ojZu8syGEAHz21OQZQWpPQpAJeyWptdl5wozXplDoVmnnc7CoIoYpRHOLR1VIDYcYgkeOzcOyhNylHhkI05oFvLXVYEVtkv9sDjFpYBYZdh2bqhCMW8RMgU01yXJoJC2B2VjR0r0bRZJam6YJMx6bwgntvFTHBFvCDQhAUxRQU3XVtFITaJbVxFGxc1axhiLrsewaTWerQirfVMZ5siTA5X2zAZVFFVFmL6/lyr5+HJqCYUHMFNw0JIcPN4cxkd0oktTaFLsE94zIZsqQPNp4bVgCFmyPkOfUGN/P3zzzu51P5/e9Mvlsa5jp39WyVQ5xAAexRSiEIBqN4na7D9ZbSodYWdhIBTpDYFMEqqKQMGFVbZKGZJgr+/hBCBy6gt+e2rS3h9/OcXly1qgktSYWsLZOcNvX27BrKpGkxcT+Wby9oZ5zuviYc1JbNtUn2Bo2eGFVEENAMG4yv9KgqINGui+K2ucW4b///W/uueeeXY498cQTFBUVUVxczKWXXkpDQ8NBK6B06LTz6tjU1DrBIq+NHJeGpqQ6StYEkzxZEmDWsiBzS0P4nTrbwwa5Tpm1XpJam6V1KrfOqyRqpsb6TCGYWRLg2CIvW8IWn2yNcO/CamYuDbAtYlIbM8lz6SyujLG0TnYM7vNf4LHHHmPHjh3Nj5csWcKdd97JkCFDuPzyy/noo494/PHHD2ohpUOjh8/ixHZuHhiVT33CYlvIoINPY1xjt6gKFHt1LumRQdKyGJBrp5dPZqyXpNZma9ggaggsC2KmRaFHJ9Ou0i3LzmkdPOS5NB45roBOPg2V1Dj/pP5ZvLsxRFnYbOnit7h97hpdv349559/fvPj1157jezsbObOnYvD4UDXdd544w2mTp16UAsqHVwC+Ljc5OHFtVTELM7rkkGPbDvhmIXPpjB9VD4NSYttEYNPt0Q4tshFvt2UrUFJaoXae3VcempbNa9NJWEKEpbAriqM+085Dk0h3jjWH06YJIUgZpioikJ7n0a6b8m0zy3CWCy2yzjgJ598wujRo3E4HAD069ePbdu2HbwSSodEU6LeekOwLWzwweYQNQ1JumfZCRuCW76q5I9fV/H8yiDHtfOwYHuU2qRNrh+UpFaojUtw/8g8Clw6hgV1cYsbB+fw4DdV1CcsAnETm6pw76JqauIW7X12QknBxT0yyLXLiW/73CIsKiri+++/57LLLmP9+vWsXr2ayZMnNz9fW1uL0+k8mGWUDoGmdUcZusrve/lx6lAdt+ipKPh0lT+PzMPvUBECVgYSFPps1MSS2FDl+kFJamW2hS2W7Ijw2PEFbKpPsjWUxGuDBhN6Zdk4sZ0XRUktpeiWqeO0KXT1O9gWTlIdsyh2pned3udAeNFFFzFt2jTKy8tZvXo1WVlZnHbaac3Pf/fdd3Tt2vWgFlI6+ApcKgUujf4Fbp5YEsCwBO28Gl39Dv6yNIDLpnBRtwzeXB+iJmbSzqtz05Acirxy/aAktTb5LpWFVQmcjgg+m4KiwqpAEocKJ7b38r8rgjQYAoeWGvZwaHDzV1UkLUF7Xz79/Vpar6Xb52u//vrruf7669m+fTvFxcW8+OKLZGZmAhAIBPj66685/fTTD3pBpYOr2CWYODCHksoG/n5qW+acXMgzJ7WljVtl5glteObENry/OUxtzKSNWydqCB5fUku57BqVpFZHU+CcLj7+sznMUQVO3tkQ5p0NIaYOzWP28iARI7VXoc+ucteCKjy6xrldffy/XhmU1iVYG07nMLgfLUJN07j99tu5/fbbf/RcVlYWpaWlB6Vg0qGlALkOwemdMhj3n3LauDVOau/lrQ0hdkQMxvfLoj5ukePUiJsWqqLQkLQoC5sMzpTBUJJak/IGiwXbI/yhXybhpCBpCjRFYX1dAq9NxWsDV+OGvaYQVMVMXl5TR0NS4LEpdMmw08Obvon09/k24KyzzuLzzz/f6/NffPEFZ5111gEVSjo8wpbGnxZUoykKwws9PLM8yNaQQYZdI2IIIoZF1BA4NJXtEQOXrlLsTfelt5LU+hS4VCJJQYbTxvLqBKaAqCGIm4JQ0qImZtLQ+Nilq82JNJoS7T9Zkt6J9Pc5EH711VdUVlbu9fnq6mrmzZt3QIWSDo9tYYOokZo2rSipfQnz3BoXdc/AqcNDxxbQyadjCkGhW+fqAdkIYfF5pWBLVGYolKTWQDT+TBqUjSXgw80hLuqegcemsLSygcePL+DqAVmc19VHpwyde0fk8XppELee2n0i362hNe5HmK4OetLtbdu24fF4DvbbSodAO6+OS1dRlVRXaTuPxq+7ZvC/K4JEDYHfoXLnsDwqIklq4xaqIhj78Q4qohbtfTp3D8uR+xJKUgsSwIflJo8sriVhCSYNzCFsCF5eU8cVvTIp8Nh4fEktDUkLXVW4dmAOXt2iNJgk06FhV1NdpumeSP8XBcJ3332X9957r/nx888/z2efffaj84LBIJ9//jlDhgw5aAWUDg0B5Dhg+jH5TPu2mg3BGA8d14ZPyyL8tmcmX2+PUJcQPPp9DRd39/HRlgj/WFvP/+uZyd9WBtkSMpj2TS29T86nnUzALUktomk9cMISZDlU6hMGtxyVw6Pf19JgCP66tJbyiIHXrmIZgoe/q+HCbj5uOTqPG7+sIN+lYVNkIv1fFAjXrFnDW2+9BYCiKCxevJilS5fuco6iKLjdbo455himTZt28EsqHTQC+LTCZH5FlP9sCXNJ9wyyXRp3LqhiU30Smwpjevt5pbSe6qhJvtuO36FQHhHETIHHrtKQtGgw5L6EktSSmtYDZzlURhZ5eWl1Pb/p7GN8n0wynDqvlNbjtqlUNpgIUj0/RV47OyJJLu6eychCJ50yNIb40zuR/i8aI7z++uspKyujrKwMIQRPPPFE8+Omn61bt7JmzRpeeeUVOnfufKjLLR2AsqjCoooET5YE2BIyqYxa3LuwmkAslXMwnBQ8szzImR192FSFtYE4Iwo95Lk1EFAdNfHYVNx6U3eKJEktoWkfwmGFbv6+uo5B+W7+troeXdNYWZPAZ1epjprNHZ52TWF1IE7PLAf/WFNHgyEQyET6+/wtFggEuOCCCw5FWaTDpCJqETMhw6ZydhcfvbJteGwqugr5rtTC2rgpsGkwtq+fdzaGUBS4cUgO/9kawhKQ4VCZenR6d6dIUksrdgluGJKNEAoxI7XzhEdX8NoU5m2LMLavH49NocijMb6vn2kj88l3aZjC5JnRhfxnUz0VEZl0+4Bu58PhcHNrcPefg8E0Te677z769+9PQUEB/fv357777sMwjOZzhBBMmzaNnj170qZNG84880xWrVq1y/sEg0HGjRtH+/btad++PePGjSMYDB6UMh6JClwqOQ6FC7pn8OLqOraEDOyqQnXUJGkJsp0axV6dPtkOPt0SxmvTOL7ITbFH4zddMrh2YDaPHVfAyXKijCS1KAU4pVDj1509ZLs0fHaFc7v6UBWoiJrEkxaTB2YxYUA272wMc/O8Cv6yJEB9UuGv31fTLcdFhwy5JGq/km7ffffddO3alfbt29O/f38GDBjwo5+D4bHHHmP27NlMnz6dRYsW8cADD/Dss88yY8aM5nMef/xxZs6cyfTp0/nkk0/Iy8vjnHPOIRQKNZ8zduxYSkpKmDt3LnPnzqWkpITx48cflDIeiYpdgoH5Tv61PkTSEujAb7r4UBWF+oSFU1c4p4uPWUtrCSUFV/XNxGuDx7+v4enlQbr4bTy+uJptabzuSJJaCwXw2wQXdPXh1BTqEhbLa+L8pouPAo9OodfOfQur2R4xMK3ULNHHv6/h0l5ZvFZan9ap1Zrs8/KJG264gX/84x+ceeaZjBgxAr/ffwiKlbJo0SJOO+205pRtHTp04PTTT2fx4sVAqjU4a9YsJk+ezNlnnw3ArFmz6NatG3PnzmXMmDGsWbOGjz/+mPfff5+hQ4cC8Oijj3L66adTWlpKt27dDln5WysFcKmC9l6N87r6aOuxsz2cYHxfP5YCOQ6Vdj4bA/Ic+GwaW+vjlDeYDMh30zXLhUbqblNOlJGk1qG8weL7igbG9cvCRCEQtxCWQFFheWUMn13F13hu004USSEIxkzKwiZ9fOldj/c5EL7zzjtcdtllPPbYY4egOLsaPnw4c+bMYe3atXTv3p3Vq1fz5ZdfMmXKFAA2b95MRUUFJ554YvNrXC4XI0eOZOHChYwZM4ZFixbh9XoZNmzYLu/r8XhYuHBhWgZCgDZulZFtPTy1LIhLU6hPWvyhXxafbglxXLGXB76tIM+tEYyZ3D0ij3ZejYe/DRNKCo4pdKb9uiNJai0EkEDh2CIPq4JJ7lpQTcxIpU576NgCMmwa9QmLuJmqqzYV3DYFm6Lgd2r4nSoCkdbDHPvcKlYU5aB1ff6cyZMnc9FFFzFs2DByc3MZPnw4l1xyCWPHjgWgoqICgLy8vF1el5eX15z9prKykpycHBTlh49ZURRyc3N/MkPOfzMBVCcU5q4LNadgynZqPLUswJV9s3l+ZZBcl4YG5Ll1Hv++Fkuo/L53BmN6Z2JTFSYNym7OaCFJUsspiyos3hEj06lz1/xqoo2p0yJJwYPfVtPZb2NcXz8OTUEB+ubYuW9EPiB4+NgCNEWkdXo12I8W4RlnnMFnn33GmDFjDkV5dvHGG2/wz3/+k9mzZ9OzZ0+WLVvGrbfeSvv27bnssssO6e8+0OThrTX5uKrprFbzqFdcxA2Lth6dUNJCBXyNSyLcemrKdYZdJRg3sIB/bwrTzW9HUSzu/6YaJRnHrsHkAX76K9VYpvFzv7pFtNbP4Zc60ssP8hoOta22PAIxHYdNJWr+cGsqgAZDsLo2gUdXeHhUPoLUWuBb51WwNZyaJDftmHwMZwOxsoqWu4hf6EA+h5/q/duvMcIrrriCa6+9lssuu4zi4mI07cezjnZvpe2PP/3pT1x99dWcd955APTp04etW7fy6KOPctlll1FQUABAVVUV7dq1a35dVVUV+fn5AOTn51NTU4MQorlVKISgurq6+Zw9OZAu09Y89rg1qrByXQOFXoHPrrE2mMASkOfScGoKSQvqEhZem0pV4/ojj66gAg9+W8NF3X0IwOlwADBrVYw5ozu3yuwyrflz+CWO9PKDvIbDwRlVWFTfQIZdw6UrRI0f6mKGXSWUtHh6WRABXNnbz0tr6shzpWZ8NxiCP82v4s3/KaJb24wWu4Zf4lB+DvscCI8++mgAli1bxosvvrjX82pra/e/VI0aGhp+FGQ1TcOyUslhO3ToQEFBAZ9++imDBw8GUrNa58+fzz333APA0KFDCYfDLFq0qHmccNGiRUQikV3GDdNFRTSVczCRNJk8KJv7FlUTNwWWJbh5WC6heJI/j8zj6eVBbCroqsLE/llEEhYZNoWkpezSH5qwZHYZSWpJxS5B7xw7QphMH5XP9G9riBsCQwgmD8rGrSuYwk+OQ6WL385HW8PoioJDUxAIooZFVYMJaby92j4HwptvvnmX8bZD6bTTTuOxxx6jQ4cO9OzZk5KSEmbOnMnFF18MpMb6JkyYwIwZM+jWrRtdu3bl4YcfxuPxcP755wPQo0cPTjrpJKZMmdI8wWfKlCmceuqprfou71ApdKsUeVWCCXh6WS0nt/eiqzCijRNFgYcW1xEzBed28dEvx8HWcJJZS2upjVtM6J9FlkNNbeWipf4NyEkzktTyir0aqwOCf66u48JuPkDhmLYuAnGDKZ9X0tajcXonHytqElQ2mCQsQaFHx4qZ6KrSuL2a3H3iF5s6deqhKMcePfjgg9x///3ccMMNVFdXU1BQwO9//3tuvvnm5nOuu+46otEoN910E8FgkCFDhvDGG2/g8/maz5k9ezY333xzcxfr6aefzoMPPnjYrqM1MQVYaDzwTSW6qlAVDRMzBPlOlZgl+J9OXlRF4attEV5aXccF3TKJWxA1Ba+W1jPrhDbYNIUGE/w2maxXklpaWVRhc73Bo9/Xsi6YZFvEQFVAVQT/3hghy6kxur2XmUsDZDpUJvTP4smSANvDBl39Nq7q68ee5osJD2gbplgsRm1tLbm5udjt9oNVpmY+n48HHniABx54YK/nKIrC1KlTfzJA+/1+nnnmmYNeviNRZdQiYQpsmkJlQyq1UqFHI8Op8fjCasJJgUNTuKqvn7gZxm1L5SfMcmhURU1K65JMHpTTmH/UxUltlLSedi1JLa0ialGXEEQNgaY2LqcwoT4hqI2Z1DbmEI6ZgliDyf9tDPG7npkoCpzczsO62hhbwybdPOlbk/frPmDevHmcdtppFBcX07dvX+bPnw9ATU0Nv/71r/nkk08OaiGlg6fApVLo0alPWM2dmad28PHssmDzIHvcFDy7PMioIg9d/Q7ipqAmZqIp0MGnc/0XFYSTgj/8ZwerQ2l+KylJLazApVLk1XHpCn67RtwQVMdMVAVsauMEQcDZuHyiOmYxZ0WQ19bWU95g8MB3tfid6Z1mbZ+/xb788kt+85vfUFdXx1VXXYUQP3SL5eTkAPDCCy8cvBJKB1WxS4AwmdA/C2fjOJ+uQsIU5Lo0GusNCVPQM8uBrsDF3TMYnGfnoVEFhBIGv+2ZSYZNocEQlIVlwl5JaknFLoGKyeRBWWQ7NVBSi+bznBoXd8/ApSv8a109twzJ5tqBWfymq49x/fw8OCqP6oYkXTJtWGZ61+N97hr985//TP/+/fnggw+oq6vjqaee2uX5Y445hpdeeumgFVA6+Oy6xkebQkzo7yduCvpkO/l4S5hQzCLXqTXfPZaFEqDA2xtC3DAkhznLa6iICgJxk1uPyqW9V0v7QXZJag08Ng1nAm47OoeEJVhWHefN9fUIARd1zyTDrpDr1nllXYBwwsKpKfTw2/nn2nrO7ZpBG3d6T3jb5xbhkiVLuOiii9B1fY+zRwsLC9M2Y8uRoCyq8OT3NYzrn81Ty4LMXh5k2jdVXD84hzYenaqoSSRp8ZsuPuyawitr67EE3PxlJQPyPFQ0GGQ7NaZ/W819x+TTyyeDoCS1pLKowjc7Yjz8XS1JSzBzaS2amloTvK4uyUur6+jmd3DrvErWBhLEjNR44n2Lqjm1g48314eoT6bv+CDsR4vQZrORTCb3+vy2bdt2mbEptS4VUYuKqIkKXNo9E5dNIZIUbA0luOWoHBoSJu0z7GyoS7I5lEztaq2AYQkcempdoaYotPXq5NoPcB8vSZIOWEXUwhBwansPpoAxfbKwY3HicQWsDyYp9OjURA18NhVf4+Q3hdTkGVWBuCHYGjbpncaJt/c5EA4bNoy33nqLSZMm/ei5cDjMSy+9xKhRow5K4aSDL9+lErNSyyheKa0jy6FRFjYY08ePrS5JfdLioe8ClEcM7I2zR7/aFqGyQSFmgM+e2s1abXyvdO5OkaTWoNCt4ndqPLE0kNpgWxFcMyiHaz7dQYMhqE9Y3DEsF5cGEQMqG1IZo9q4NXw2FZdNriPcr3WEZ5xxBueee27zurySkhLWr1/PzJkzCQQC3HTTTQe9oNLBoSlwafcMqhqSPDgqjw11SQo8NrLtKu0ydK7+rJIuGRq3Dc2lPmHitaU28b1Q15i1tJZAzMTpURjTJwstfW8gJanFCVLdouVRwewVQbx2le1hg/H9svjT/CryXDqqIsh3azy7LMDdI/K55rMdZNhVIkmLC7tn4LUpTBmUQ5Y9vW9o9zkQDh48mLlz53L99ddz9dVXA6mcoACdO3fmtddeo3fv3ge3lNJBUxWziJrw/uYwZ3fO4K0NYbaHDfJcGtcNyqbAqXB6pwxu+KKCuClw6grTRuaxcHuE87tn0GAI+uc6qYwkKW+wKHLKaChJh5sAPiw3eWRxLce181LTmBdYVcChKbh0lYoGA49NZWvIQFNgcWWUB44pwLAEJdVx/rG6jjuH5+HVBdsjFm0d6VuX92tB/ahRo1i0aBHLli1j/fr1WJZFp06dGDhw4GFLvybtH5umMaskwG97ZvLM8iA1MROfTSWcFDy8uJYHj83nyo/KiTVmsY8ZgtvnV3HviHzuX1SNS1fw2VQcKjK1miS1kLKowiOLa0lYAl1JDVnUJyxURaFLpp1QwiLD/kPifE1VMCyF67+oYM5Jhby9IUQkabEmkOCEdh4KXOm9qdoBZZbp168f/fr1O1hlkQ6DaNIku3EjzqQlyHGq/KGvnx7ZTkJJE2GldrDXFUABTVGImalsMz4bXNUvi1yXSq5DkanVJKmFVEQtElaq/i0sj3BV3yyeXR7AripUNSS4c3gus5cHqW3MJXplXz+fbQ3TpnHbtUt7ZOK1Kby3KUT/XCcjstO7AbPfgXDTpk1s2rSJYDC4y6L6Juecc84BFUw6NApcKrkOlRyHRpcMnf/p6EXXVS7/cDtRUzB5YDZ5bo0dEQNLQNwSuPRUJfnDgGw6eDUK3DYGZloytZoktZACl4pdVUhYgkDcwqkKnvhVPoGoSb0B07+p5rpBOSyviePQFN7dUI8hFGpiJqtq4/zfpjBndvKhqwo5ToV5VSaj8rS0rdP7HAg3bdrEpEmTmtOq7SkIKooiA2ErVewS3DAkm3XBJBMHZBOMW9zyVWXzhp6vltZxSY9MXl1bz9awgVNT+NPQXF5cGaAqbvH48QVk2EjbCiNJrUFTPW7qHvXZNeoSEEjAX5fWEm7cnf6crhnMXh6k0KNTHTW4oo+f10vrMQQ8tSzAI8cWEDehJpbqbm2N+4oeDvscCCdNmsS3337LlClTGDJkCBkZrXszR2lXCnBKoYZTU9kaMYmbonk8EKA8YvLqmjoe+1UbSoMJHJrK/66opSYmsGsKOxoMkpZGJ7dcQyhJLaWpHvcenU9F1CKQFISSqWQYTg3O7uLDqSlk2FSuHuCnc6adxRVx3iitpzZuUejWcLg01tclENiIG6T1vqL7HAgXL17M5MmTD+t2TNLBJQAT8Nk1BCbO3Xa1DiQsYqbgz9/UEE5a2NTUBBm3TcGw4O4FVVzVN4uLOuoyGEpSC1GAdi5BW5fC22UWuqJS6IHR7b38ZUkA00qN7V/Rx49dU3hxTR3xxkX0FhBKWPTIspPvSu0sk86T3/b5e6yoqIjMzMxDURbpMFkbVqmNWby8KoDPBncPz6XIo5Pj1Cj26tw3Io/Pt9Qz9agcir06RR6dArfGdYNy8OkK1w3MYsZ3NaySO09IUotbVq9SUh3nnoWV5Dp1XlsbSj2hgCHgvU1h1gXjjO3rx6Ur5Do1AjGTW47K4fkVQZyaQhsXaT35bZ9bhNdeey1PPfUUl112GV6v91CUSTqEBPB9dZJ1dXF65DjZUJck26FxUXcfSSu14D5uCo4q9PDk0gDndfVhCuiR5WDm0hrCScH1g7L5fc9MKhpM+qRxWiZJamkCKA0mqU8IGgzB9gaDmGmRaVdRFXDpKklLkO20YVlJHjm2gFW1McJJqGgw2BoxqIganFWUvhNlYD8C4e9//3ssy2Lw4MH8+te/pm3btmjarntZKYrCtddee9AKKR08ZVGFmSW1jO2byj5xz4g8pn5V2TxOqClQ4NY5t6uPrWGTv6+qpzZuYlMVLu2RyfMrgzzyXS03Dslp3MMsfdMySVJLK4sqrA8m0BQ4v2sGHptGMG5hWIJCj86m+lSu0bXBBHOWBwEo9upURk1+1zOT+riF36GzLUraTpSB/QiEy5Yt45FHHqGqqoo5c+bs8RwZCFuvisYd6reEkmTYVdYG43jtKolYKthl2FU0FSwEDi21jjCb1NZMLlvqDjRmChKWaNzDLJ3vIyWpZVVELdbURPl1t0wEUN2QZOKALJ5eFkAAnTN07jsmn7ghOKmdh+XVUT7YFOHsLj7eXl/PlX39bA8n8aha2k6Ugf0IhFOmTCEUCvHYY4/JWaNHoAKXiltTiCQt3DYFFQVLCIq9OkkLqqIG+aqOYYKqKFTHTKKGwKkp9M52ku/WcDQGTJl0W5JaVqFbpWeuiz8vquaqfllsqEvy3oYQV/b2c3yRix1Riwmf7CAYt3BpCneNyGXK4CzWBBKc0tHHV2URemRlpX1d3udAuGLFCqZOncrvf//7Q1Ee6RArdgmuHZTNUyUBxvfN4p9r6ph6VA5PLw8SMSzyXTq/7ZmBTVWa069pisXYvlk8/G0Vl3bPoMhrw6kJmXRbklqYKeD1dSFOaOflz4tq6JdjY+rQPBoMC5dN49aPygklUwEuagruWlDNo8cVEDUEn22NMHFAFg7FojwmgNQ6wnSs1vscCDt27LjHRfTSkUEB8pwK1wzKxrAEY/tkoKgqp3fwEjZSU6tV4POtYX7bI5OufhulwSSvr61DURWGF7rYEU4SNxSZdFuSWlhlNJXhyakpZNgVRrb18Mh3NSQtuHpgNjEzFdiavrGjhiCUsOie7WBGkYuqcJKoqfLVtiivrQtx97AcTilMv4kz+zz//dZbb+XZZ59l8+bNh6I80mHgtevcv6gGh6bid9q5d2ENL6+p539XpHasf3RJgIEFbv65to41gSRzVgTZGjEJxi0CMQtFVfE79cZ1R5IktZQCl4pNU1AVhbO7ZPDymnoSliCStPDqSuNG2j+c79QUHJpCwoSN9SZup50/f1NDrxwnW0IG076ppSyabmFwP1qEX3zxBZmZmQwdOpTjjjuOoqKiPc4affjhhw9aIaUD17R3WUXUIpg08emQNAVRYVHRYJDj1MhzaegqnNMlgz45Dv5+altmfl+NAhR7NG4blupyKfLaUIRJsUv7uV8rSdIhVOwSTOqfzXMrAlzZ189bGyDHqfLrzh62hZM8M7qQv3xXTd88N7oKo9q6CUSTmKRakA1Jg+qYybZwEiGgwRBpmWFmnwPhc8891/z/H3/88R7PkYGwddl577LquMV5XX2c1MHHPYuqeWhUPg4tNR7YM8vGSR18zCoJNG7NZHHviDxO6WiSFCp/nFeJo3EbpluPymnpy5KktKcAg/NsdByRS10itSXTMW09zF4exBCCXIfGH4fm8tLqIFVRi4+3hLmwWwYJS9DF7yDTDm1cKjZNRVHArStpmWFmn/u2AoHAz/7U1tYeirJK+6lp77KwISiPmERNmLuunvqERYNhcvuwXJy6yqgiL7NKAnhtKoG4ScIU3PZ1FUVeB9O/rcGlq3h0FUvAX5akZxeKJLU2Hk0AKrfNq+SGIbnMWREkZgpMC5w2ldu+rqR3jovNoSSbQwavlYawqwp/nFdBXVxw78h8nl1WS7FXZ+rR2WmZYWafW4RCCLn57k+JNVDw1bvYVufudFDZ4//S/HdUdnu8Bzs/96PX/cS5KOgNgnM2hElYCrVxi6MSLtw1MTRFJe9zN2UVDXzR0Ud4o8CzPUyDYWEKJbUfIdB+fibX7AiTWjWY2qhQVUFVvOhe9Sd+94/L8rN/g729z8/9DfjxQ/+OCvTarbscFMqe/mZ7+917+Nz2dD27v25P770f7+nbtg0tXrfb4T39DXZ/jx+IPf7evfx73FN59/Zv8kfHd39d6j+u7VtR9Z2SLvzkv4/dH+/5PXd/IPblPff2+f3EZ2YPVKFU7SWL1r7Uy1/62f3EY7GHMlcHobrBIhysY8t2QTvRgNDBriooiTiRuEmesNPXlcSwBMlonGREgYYIZkMYRWiM7+qgX46Ngf44SmwP17K3su1+/BfUyz3XhV/wHXgIKcFgcJ/Cf+/evbnwwgu58MIL6d2796Eq1xFLCdbgue68li6GJEnSf43E6N+wYsSZdOvW7ZC8/z53jQ4ePJinnnqKUaNGceyxxzJz5kwqKioORdkkSZIk6ZDb50D44osvsmbNGmbMmIHP5+OOO+6gT58+nHfeecydO5doNHooyilJkiRJh8Q+d43ubsuWLbz22mvMnTuXNWvW4PF4OOuss7jooos4/vjjD1Y5jxzRBsIvPEF2dnbq8S7JB8Qe/lfs4bw9PN79ePPzu79uLw92el1tUuHdjWESpsBrS2WpL/bqbK5P0inTzqa6BA2GhQA0UmPC3f22VI9+4/vYVIXt4SQnFDnxaj9T1r2VV+zhvL2Ve0/v+5Ovg3A4hNfzw9iOsre/9c++t9jlPz/5u39xeX/+PRsaIrjd7p9+zz28Ttn9vff6b3Bv/3Z+ory7l3mX8378ungshsPh3PMLf/S6X/aeO5+n7Nd77vbEHt/7h9+bTCax2Wx7eN0vLO/u5+7xd++hvLu9j7KH15kCEqYAIVAVBVMIbCoYlkAjlRd497K4dAXTEhgWODRQG3eccWlK6nfs/jfYQ1l+uh7/xOv28N7KL0jQcqi7Rg84EDbZtm0bd9xxB2+++WbqjRWFtm3bMnHiRMaPH/+jtYb/zUpLSw/ZB3YwfFBu8fuPdpDtVLmgawbtM2zku3X+8l0NI4s8/GNNHYG4SaZdQ1HAEnBRDx+dMuzUNhhYwOdlEcb2zeK89q03C0Vr/xx+zpFefpDXcKgJ4NMKk8oYLKuJ8cmWCHcOy6EuCatrE7y8po7amEnSSk1NyXdr3D08l3sWVlMZNXn6xDaUBRromedufRlldguQpevWHbLPYZ9nje4sFArx1ltv8eqrrzJv3jw0TeOMM87gkksuwW638/zzz3PbbbexatUqnnjiiYNVZukACCDDoVHo1jinawavldazLWLQ1qNzXlcfLi2VfSLLkdq1GsCpK/TPddIpQ6cspLKiJs5tw/Lo5NnDHbkkSYeNApxQoPF9ncqa2gRCCALxVFC0BESSFtlODV1R0FTw2lQ0RaE6auLWVQo9Oj6bm5PatLIgCId1Buk+jxGapsn777/PFVdcQY8ePbjmmmsIhUJMmzaN1atX89JLL/E///M/nHLKKbz88svccMMN/Otf/zoERZf2R1lUYVVNjLtH5vHK2nq2hQ2EgNqYyevrQsQMwaSBWTQYqSnvdk3hij5+/nd5gM+2RliwI0aR10bSMFpfxZGkNKQAhS5Bv1w7tw7N464FVSwqj5JpVxnT208wblEZTdXzy3tnsiGYwKmr3DsijyWVUYIJ2Jbma4L3uUXYvXt3AoEAbdq0Ydy4cVxyySX06NFjr+f36tWLcDh8QIWUDp6KqMX2BouaeByblspFCJC0BNVRk545Dlw6TB6YRaHXTtSwqI0ZWAIKPHYG5Gq8tCpAbb6LDJsqk25LUitQHbXIc2lUx0wcukJF1OSl1XVc0C2DB0fl49IVVtbEqYtZ9My2M+ekQtYF46yrN3HoelqmVdvZPgfC0aNHc8kll/CrX/3qFy2sP++88zjvPLmurrUocKk4NHDrKs7GbLyWENg0hRynxuLKGPlOjfqk4NEvK2gwBG5dYWw/P5l2hfqkydawQY8ckfZ7mElSa2ABywImM76r5ZqB2VgiNVkmELd4dEkAh6ZwaY9MPtoS5oJuPuy6yvZQnGyXTmbUxKFBxEwNcqRrKNznQPjMM88cinJIh0mxS9A3247LrnF570yeLAlQ0WCS69QY2yeTIq8NFFgdSJDjVEk0mERNwdvrw7g0haQluG14PiurY3I/QklqYQJYUqfywLc1bA8bvFEa5Lajc/lrSS15Lo1RRW56ZzuwqQr/09GNXVPYHDLok+tieyTJ6HYedMVi1tIAnYdn0y4N06vBQZgsU19fj2VZP3quXbt2B/LW0iGikNrVem29SThhcvvQXCKGoFumzqraJNd8XoHXplKfsJjUP4sPNoeoTwpihkV9wiJpQU3UZN62Bo5p45Bdo5LUQpqS6W+LGEQNQbZLY0Shh+2RJOd09uJ1aDy3oo6/razDrincclQOr6ypY00wSZFH59yuPl5ZU8XEgdkcW+ymJp6+3aP7taHc888/z5AhQ+jQoQP9+vVjwIABP/qRWq8ch0qWQyPfbeP6LypYF0igoHDH/CoSpsBqXJs0syTAr4q9lIUMPDYVm6rg0hXWBuIMLHDJ/QglqQU1JdP32VP18qxOPqrjFjO+qyWUFPzl+wClgQQJU6ArCncvqGZwvhu3rrI1bPD6uhAntfdx14Jq3DYdS0mfJW672+dvshdeeIEpU6bQoUMHbr/9doQQTJgwgSlTppCfn0+/fv3kUolWrtglcKgmhV6dK/pk0SvbTkXUwLBSKX3rEia5Lo2EKRAIfHaV6wZl0yfbRm+/zgebw/TJsadllnpJai0qohYJS/De+jpuGpKD15baGSZqCGKmQAhBgVujrVfHbVOwhMCpp7ZaUpXUOGKmQ8Vo3Mi3pnG5VDra50D41FNP8atf/Yo33niDyy+/HIBTTjmFO+64gwULFhAMBqmvrz/Y5ZQOsmAc7vi6iplLa5n2TQ0eXcOlKygKmBYE4yZtPTrD2ri4qHsGeS6VhqTARCHXqTEoW0/bgXVJag0KXCp2VWFjyGBAjo0zOnrIdqq4bQqZdhWvPRUYt4YMGpICTVVQSK0hzLCpKEA3v50+2XaynSpFHtki/MU2bNjAGWeckXqxmnp5MpkEwO/3c9lllzF79uyDWETpYNsaVXjwu1rKIwZFXh1DCB7/vpqHji0g362T59JwaAqTBmQxY3E1dk3hpi8qsesapcEk1wzKSdtBdUlqLYpdghuGZGNXFSKGwKEKjs53MPXoXHpk2bm8t59QIpUqMWpa3DE0l/c21uOxqTQYgimDc3h1bZA/9M/Gb1fIsKVvnd7nyTIejwfRmPrG6/WiaRrl5eXNz2dnZ7N9+/aDV0LpoBLA0lqD+oSFQ1MoCxvYVYXNYZOtoSQXdvMhhMLQNk421cUZXODhtdJ6KhtMIkkLm6oQSZpYaKTv/aMktTwFOKVQo+9J+SwLGqyPCR7/rhabCtcMyqE6anBxj0w0NdXL8+LqIGd0yqCDT2dNMMmm+gSDC9xUhJNYlk6mTaE4TSe/7XOLsHv37qxZswYAXdfp168fr7zyCslkklgsxiuvvEKHDh0OWgF37NjBH/7wB7p06UJBQQHDhg3jq6++an5eCMG0adPo2bMnbdq04cwzz2TVqlW7vEcwGGTcuHG0b9+e9u3bM27cOILB4EEr45GkLKoQN8GhKVTHTCyRSrtUFzcJxi3+ujTA2xtCXPlxOaV1Jv+7Mkhlg4lTU/DaVDpm2nFoGkvr5EQZSWppCqlcwIqi8cSSAIG4RWXUImEKIknBy2vq+N8VQf53ZZBVtUne2hBiVSDJi6vrCMYFzy4PUuC10dan47Kl763tPn+bnXnmmXzwwQfEYjEAbrzxRr7++ms6duxI165dWbhwIVOmTDkohQsGg5x66qkIIXj11VdZuHAhDz74IHl5ec3nPP7448ycOZPp06fzySefkJeXxznnnEMoFGo+Z+zYsZSUlDB37lzmzp1LSUkJ48ePPyhlPNJURC22hpJc2cffmG0+FRQv7+3ny20Rirw2YqbV2D2aeo1TU7hrRC4KJpoi2FIfpyycvgPrktSaVEQtamKpHptw0qKtV6MuZnByOxd3DculjVvHpSu08+lc2iODxRUN3Dksl9LaKNkODbsKDQkTy0rfOv2Lu0ZjsRjvvfceyWSSm266iWAw2NwCe/fdd3n77bfRNI3TTjuNUaNGHZTC/eUvf6FNmzY8/fTTzcc6duzY/P9CCGbNmsXkyZM5++yzAZg1axbdunVj7ty5jBkzhjVr1vDxxx/z/vvvM3ToUAAeffRRTj/99FadVf5QKXCpGEIQjlv8vncmNlWle5adR7+rJpwUBOMWlhB4bCqnd/QytI2TTLvGxro4OyKCuqTBUflOOng1BJacMCNJLSzfpRJIKjg1hUyHyq87+aiKWTz48Q46+HQu7u6jR7aD9l6dj7dGGNrGzayltZzdNYPtG0O4bSqhhMmGkEXfzFaYfPsw+EWBsLy8nDPOOIPNmzcjRGp/OrfbzT/+8Q+OPfZYRowYwYgRIw564d59911Gjx7NmDFj+PLLL2nTpg2XXXYZV111FYqisHnzZioqKjjxxBObX+NyuRg5ciQLFy5kzJgxLFq0CK/Xy7Bhw5rPGT58OB6Ph4ULF6ZdICx2CYbk2SlvsMgTOrd+VUkHn86YPlk88G0NphDYG1uI13xSzkkdfHy0JczvembS2a8TDySZOq+SJ08spCyqyEkzktTCNAUSZpLrBmWxoS5JRdTk76vqMIRgZW2SlbUBumfZObHYzdsbwmyPGAhg9vIgDx9XwF+X1HJyew/vbggxOCc/Lev0LwqE9913H1u2bGHixIkcd9xxbNiwgYceeohbbrmFr7/++pAVbtOmTcyZM4eJEycyefJkli1bxi233ALAuHHjqKioANilq7TpcdMEnsrKSnJycnbJi6ooCrm5uVRWVu71d5eWlh5Q2Q/09YdSB01HdRVRb6ncNCSb/jl2HDaNB0flk+VQWR+I8/r6ELqucmyRmzM7eRDCImEqfF4WZHuDSXnEIBkKEUtWtfTl/KTW/Dn8Ekd6+UFew6G21ZbH6nobR7VxkuN0sXBHagsmTVHQGjfeTVqCjhk2zunq46PNYWrjFkLAqtoYZWGDSFIQikZZVxFo1XX6QD6Hn2r0/KJA+Nlnn3HJJZdw3333NR/Lz89n7NixbNu2jaKiov0u3E+xLItBgwZx5513AjBgwAA2bNjA7NmzGTdu3CH5nU0OpKXY2rtcBVATAJEUtHWrrAsZ3LdwB+FkapfqO4bl4tIU1kUN5pU3oCvQK9vBxroEg9q4aTAFhR6dDu4s2rn8LX05e9XaP4efc6SXH+Q1HA7OqMImI4ZdVwkZEDet5l3nHZqCz6ZSHTUpDSb5x9o6Lu/t5/XSegIJi7gBLpvKgDwHSys9dC1ovXX6UH4OvygQVlRU7NK1CKnuRSEEZWVlhywQFhQU/GiLp+7du1NWVtb8PEBVVdUuuU2rqqrIz88HUgG7pqamuUsXUmOL1dXVzeekm7KogmEJDBQsRePuBRXEDIGupu4c//xNNbNPakvUMMlsXJRbFkoyb3uEZTVJHju+ALsqKHK19JVIklTsEvTMdmIIeHhxJf/Tycf4fn6eWhbEa1OJGoLLe2fyny1hTAteWFnHFX0yyXfr/GdLiCt6ZxIzBRMH5qRttqhfNGvUNE2cTucux5oeN80ePRSGDx/OunXrdjm2bt265qDXoUMHCgoK+PTTT5ufj8VizJ8/vzlwDx06lHA4zKJFi5rPWbRoEZFI5EfBPV0EExbr6g2+KGugPmESM0TzZkqmgHBSsDWUZHVtgsUVMUqq49Q1JtzOd2msDyZYUB7jo3JTbsIkSS1MAWJJk4oGg20Rk7ml9Xh1hemj8rl7eC6X9crkldJ6toQN2nh0nLpCkdfG2+vrOaNjBoYpCCUFHk2k5UQZ2IdZo5s2bWLx4sXNj5vSqJWWluL1en90/pAhQw64cBMnTuSUU07h4Ycf5txzz6WkpIRnnnmGO+64A0iN9U2YMIEZM2bQrVs3unbtysMPP4zH4+H8888HoEePHpx00klMmTKFxx57DIApU6Zw6qmnturujkMpicbDiys5u7OPDHsqi0zSEphWqtvUqSnYVIW/Lg1wSc9MADRgUL6bBTsaCCUtumU5eWRxLb1Hp+fguiS1JllOnaSwcOkK1XGLV9eFWBdMMravn3c3hqmJmmQ5NLaFk6iKwqraBN9UxtkeqWX6qHw0RcFlU0ntbph+fnEgnDZtGtOmTfvR8ZtvvnmXx01dkLW1tQdcuMGDB/PSSy9xzz338NBDD1FcXMwf//hHxo4d23zOddddRzQabV7SMWTIEN544w18Pl/zObNnz+bmm29u3iD49NNP58EHHzzg8h2pqhoMFCHIcarkOhUeOCafPy2oIpSwcGkKdw7P5W8ra4kYqZ0oAFBAV2Fi/ywCUYNtoSQJS6T9ztaS1BoYpsn7G+u45agcZi4NkLQEXTJtFLo1ftczg2eXB/HYVKKmxVV9s3hlbR1CpFqT4aSFUwPLSt+teX9RIJw5c+ahLsdenXrqqZx66ql7fV5RFKZOncrUqVP3eo7f75cbCu+ko0/nwu6ZPLeyjooGk/7ZNl46rS2rauLYNJVnl9dSHbXw2BQ0JVU1PDaFThl2embZmRc1yHIo2FWlcSsm2SKUpJZU6FbpkOnEpVrcdnQOX2yPkmlX6Jhhozxs8NjxBVREDNbVJfnnmjoskdqFwqmnkmosrYojhEOuI/wpl1566aEuh3QYuTTBOxvDlEcMTAs2h00m/qecSQOymfFdDTFT4NIV/jQsnxynRmUkSU3cwm1T2FQX44VV9Tx8bD43DMlO28F1SWpN6g2FUNIiw+lkzIfbiZmCcX2zuOazCq7q62drxCDLrjC6nQeXBg0mrKyJ8buemTQkTL4oi/BlWSRt1xHKhJFppinpdtRIZYVR1VSuwrgAuwZndPRydhcfp3TwkjQt7llYiaKp9MlS0RRBZdQiZlrUJwSnFKbn3aMktSYCWBNMkmVTCcZNEhb4HRpRU+Cxq+S6dd7dECJkwOUfbee1dWHe3xTmdz39FLkFDabAa1OahzrS0T7vPiEd2ZqSbgtAVZTGzXfh7M4+Hv4uQFXUIGmlukMLPTq/6eLjlq8qef3MYj5eH+LMTl589hDtfRpKmg6sS1JrUhZVSFrQNduJQJDv0igLGwD8urOPJ5cGOL2jj2eXB4mbgphhkOPUuG9RNc+cVMiOhjBHtXHz+dZI2g51yBZhmmlKun1x9wyKvTqqAjHTYmCeg0A8tRxCAfJcGrUxE5uaWpBbEzNZWR2lImJwzYAsCl0iDauLJLU+FVGLzXVJQgmTWUtquHFIDsUejRyHwtACJ0II3LqCKVJBstir47Wlcg5XNiQxRCprcDoPdcgWYZppSrq9pCLKo8cXsLImToZdo0OGTq5TQwAJU1AbN9EUhVDCwq4plAbiDG/roWOmjZqGJL//sJIbhmTL7lFJamEFLhULQaZD4/vqBOd1s7huUA5PLgtwcntBKJma+Fbs0WkwRHNr0WNTcOsaOQ6N0e3c9M9I3yT6skWYZpqSbo/pl8Xd8yswBSypijH1qwom9M9q/gdhUxTG9vXz4eYQ53b1ETUs/r0xTDRp8NqaOhKW4JHFtZRF07XqSFLrUOwS9M+xU1Yf5+4Redg0jaeXB9gaSvLm+hC/7+0nbgqu6pdFfeOO9XZNYUxvP49/X0PnTDsONV0XTqTIFmGaUYBfFWi8Xy6oicOHm0OM6ZPNvzeG+XZHhIeOK2DRjihdMu2UBuOc1snH66X1nNctlYGissEimEiNDcp1hJLUsgSpMUKbptA924nPJtgYsggnLLIcGtl2FZcGnTLtWAIu65XZPLL/5rp6VFVhWU0cl6bQ05u+9VgGwjQjgI/KTdYGkthUWFmbJJy00BVBW5+Dj7dE+PuqOjLtKsGE1Zy4N5wQKArsaDA4vp2Xz7eGiSRF2g6uS1JLE8CH5SaPLK6lPGoRjJvcNzKfbIeK26YSTlqM7uDliaVBMu0q53b18X+bwlRFTeJmqs52y7KT5VAo9mqka1YZkF2jaacsqvDI4lo0BFf28ePWFb4tDzFtVAFzVgRZUB7hjmG5ZDpU/A4Vl/ZDF+mZnbzku3ReXlPPyLaetB5cl6SW1lSX6wxB0hJc1svPyto4+W6Vy3pm8Mejc3lzXYhOPp2Lu2eQZVeZ0D+LPJcGQLFX56xOXgbnu1CEydaokra3tLJFmGYqohYJS1CXBJE0mNAvE6dNZ0lljN5ZNn7VzsuzJbVMHpLL2mCcPtlO1gXjnNzBx9zSeiYPyiZpwbA2To7NU9J6XEGSWlJTXc7QVU5q7+X5FUFMAbkODUMIqoIJhBCc1MHHnBVBEpagyK0xfVQBlQ0m28MJXlpTR7HXRiRh8mVZmCv6+tNyApxsEaaZApeKXVXQFTAtBa/Dxh3zqwgkLH7VzsvMpQE2hU2WV8eZs7yOyZ/v4MXV9fzvyiD1CYu2Hhtum0Inr5p2lUWSWpOmunx8OzezG9cI5rk08jw6f1kaIGYIzuycWj8YMVJtvc1hk5u+rEQA/ywNUdc43v/imnoGFbjSdgKcDIRpptgluGFINt9XNNApU0NVUkl3P9wUonuWnYSV2orl3xtDTOqfhda4h6PXltqwd2NdnGsGyC5RSWppTXVZRcESqXW9mgpb6pNk2FXe3xSih99Owvyhrua5NCqjBg5N4ZwuPmadWMjrpUEaDAtDpG92Gdk1mmYU4JRCjV7+XDZGBDZVIcuh0t6n09atUejWiZkCw4I319Uxvp+fo/Jd1Cct/m9DPVf2y6LYrcqsMpLUwhTg5EINn93Na+vq0dVUpqhQ0iKStMh2agTiJkVenXhj/uDKBhOXnmr//HtjmHc3hvl1Zx9WeQO6ItI2kb5sEaYhBQgZCl9tb0AIwb0j8jm/u58751cyaUAWkaRFdcxkR9SiQ4aNGd9V8eh3NQwqcBGMmzjU9KokktRabYsqPP5dNRd2y0ABKhpM3t8Y4o9H51ITM3lxZZCLu2fg0hW2Rwx0FaYelcM/VgfYHjEIxi3mrqtnbL8svq+Ipu0EONkiTEMCqI5Z5Dh1auOCNi6ImSoj2npRhOCVM4qobDDIc2v4dCg+Og/DEkz+spJbhuRSGbVoL9cOSlKLq4haVERNFpVHuH9EHpUxi/q4ia4Inj+5LXUJk3Y+nVM6eNgaSpLv1nn0u2qWVCdImgLFlso5bFcUHh6VS7ErPXepl4EwzTStPbr/m1rKG0yG5Nm5sHsmlVGDt9fXc3bXDO5/fxtuXSViWEzqn4VXV/DZVRwK+Ozp2XUiSa1R04QZgB1Rk9u/rsJjU7ikRyazllVSHjHIdWmc29XHR5vDHFvsoTRopHKNWhZCgF2FThka7VzpO9whA2GaaVp7pCDIsKuc3y2TFbVx3l4f4taheTz8bTV5Lp2kJfDadJ5dHuSKPpmUR00eGFVAjlOhKA27TiSpNWqaMFNSneC+hVVkOlTO6uTjqZIADYag2KunJtIIuHZQDpoCBU6N51bVUeDRcWoKUwZl0cuXvkEQZCBMO01rj0JJgQbETYEl4NSOPjbWJUhYsD2SbNymCQrdqcoiUFhaFePfm0LcOTQnLdcaSVJr0zT5zak5ea1UwxCpehu3UrNIs50qJ7bz8mRJAK9NpSFpcfeIXO4bmUttVLAtksBjk+uB5WSZNJPq1lTY0WByZmcfpcEknTLszFkRZFC+k5qY2dzpKQTUxEz65bro6LNRE7OojVlM+yY91xpJUmukAJ19Kn67gtb4je7UUsHtxHYeniwJNC+hiJmCP82vxqlprK9L8MSSANO/lfVZBsI0U+wSTBqYjV1VMEVqiYQAMuwqVQ1JJvTPoq1HJ8ep0dajM7F/FgnTpNCjke9SOa+bj5M6eKmJp3dXiiS1JsUuwfVDsrGpCvO2R5jQP4sumTYMC6zGfQjrEyYAhhCEEyZvr69HAeJGeq4d3JnsGk0zCjAyX2dSfz+DClx8uCXCxroE2U6NDLuGUzU5o5MXS4CmgEOFmCG4a2EVZ3b08q91ISKGoF9OPgP8sntUkloDATQYgmsHZuPRwW1TsSxBZ7+DAreOaQk0RUGoAreu0sajY1NBUxXcupwAJwNhGip0CgYXuAjEDR4elcfWsMHQNi62h5O8UhpiY32y+dxOGTYu753B5vokzywP8rsemXy8NczMJbUMzsmnnZw4I0ktblVI5c+Larjp6Fz8To0bv6wkGLfonpHq1Zm5NECeSyMYN7ltaA6BeJKbjs7jhZUBJg/MSsu1gzuTgTDNNG3DNLMkyIhCN/csrKGiwcSmwvRR+QgEnTJsGI0zzcojBtUxC11VMIXAoYGuKHIvQklqRSoaDE7v5OP+RdVMGZRNJCnQFDi60MOspbWc1sGH164yON/B7GW19Mhx8XlZhCmDchgtJ77JQJhumpZPHN/Ow8tr6tnRYJC0IGFCSXWchAkx00JVoDpq4tAUtMZa4lAV4iboGmmbikmSWqMsp86cFUFyXRqFHhsAp3Xw8dSy1DKK51YGyXFqzF5ucWnPTAwLTAtu+aqCnlnFDM5M7zFCOVkmzTQtnxAoxM1Uqw9S4ezN9SEmDcjCqSmEEhbtfDpXD8ji620NFHt1/tA/i++qGsjUlbRNxSRJrZFhmuS6NLy6QkPS5NqBWWQ4VGKmwKamkm1HkhZRU9Ddb2dxRQOmEMRNQVnYbOnitzjZIkwzO2/D5NQVFIXmRl1d3KI2ZjDjuAIihsC0LDbXJRja1s3AXAf5Lo3T2zkpcKlpm4pJklqjNi6VXpk6/Qvc3LuoGreu8Id+2XTOsJG0BLqq4LGpGJagJmawOpAg16VT6NFp70vv3elBBsK005SJ4rnlQX7XM5MXV9exo8FEBa7q68elqWwLJ/jLklq2RUwMC3QV2rh1LuuZwcVd3XKCjCS1MsUuwcSBOVz+UTkZdpW1wST3flPNRd0yeH1diO2RJG09OpMHZfPEkhoEEIib3DU8l1y7rM8yEKaZpkwUvf25lNZbdMvMJWoK8tw6Li21ELc2Lpg0IIfqqME/VtexI2qStAQRAzaGLYpdMhOFJLUWgtTYf9w0yXdrxM3U+F9t1GTuunpuH5rHmkCc0e3cANx/TAEOLdWFWh422NFgUexM7xotA2EaUqCxVady41e1DChw89C31fyup58GUzTnKSzy6lzaM5OXVtfhtqmoimDhjhhRwy5TrElSK9CURP+RxbWc2N5LbdQkw6GBAoaAQMziu8oYJVVR2nl1Hl8SYFvYwK0rTOifRddMnUK3nPQmJ8uksWKX4LrBOYQSFtcNziViWDy9LIBLV8l3pQLdG+tC3DE8j2sG+DmqwMHX2yM8slimZJKk1qBpFnjCEiwsj3DTUTkEYiaFHh2XrnBlXz+LKxqYNDCLrWGD0zp6mTggi2Kvzj/X1rM9YmKmdwwEZIsw7W2LmLy0pp4Lu2fg1MCtq1THUmOGpoB8l8bGYAJD/HDPKNcQSlLr0DQLHCAQt0gaqeURmXaFkW3dVDQYHFvoZHPYYM7yIHFToKkKY/v6mVtaj9+hUxuX+4vKFmEaK4sqzFxSS7LxlrC730F9wkKF5sBXn7Ao9tmZuTTAxjqDYYXundYQSpLUknbejxAg32PjuRVBKqMW64JJrv2sgh1Ri5lLAxgiVa/jpmD28iBndfZRGoxj07QWvILWQX6bpbGKqIVAkO/W+LY8gkuHsX39OBpX0Du1VNfKhro4phDETIEQcg2hJLUWTbPAm4JheSjJxP5ZaAoEY0mmjyrArinoikKeS2v+wk+Ygh5+O9/uaCCalOsIZddoGst3qURN8NkUftXex6raJPO2RxjXz08oKTAteGtdatd6p6bgsSmc3dlDL58lJ8pIUivQPAt8dD4VUQu3TePuBVWM6eunvMHixi8ruKRHJuUNBm09OnluDSHAoSkEYyYKyAxRyECY1hpMhSv6+Flfl2RWSYBOPp3f9fbzwDfVuHSVcMLiir5+3t1Qz61H52KaFpk2uZBeklqTplng7VwKglSdbTDgzgXVJC3BvzeGuKKPn7+tDJLt1AjGLS7unsGrpXXcPjRX9u4gA2Fa2xwySBomQ/KdvLcxTMyCmqjBBd0yyXKqHF3goiKS4Kaj8tAVi+eWhxnZxkFRmq85kqTWqqmF+O52C9MSZDs0EPDZ1jCX9shkZKETn12jpCrGeV19dPCq8sYWGQjTWscMnY1hg+8qY1THTCwhSFrw8eYQv+6awZUfbafBEGTYVW4bmoNL7lsmSa2eALKdKjkuja0hAwFURU1CyTBHFzhZX5fgi7IwkaTgt908yPosJ8ukNcOCp0sCfLUtwti+fjRF4c119fxxaB5/WxkkaqQS9rp0hVlLA1wzKIeauMXWqCKrjiS1UqtCKrUxk/O7ZWBvnPhm1xTO75aBQ4Pp39Ywsq1HTnrbiWwRprGysEF11MRrU5m3PcIf+vkxBbhtCtlOjUw7GEIQSli43TrfV0Z5Z0MIu5qaOSqzy0hS67MtYlAesfjHmjrG9vFjCYGiKPxzTR1tPTl4bSpD2zg5Lk+mSmwiA2Eaa+fV0RSFigaTHQ0mZSETh67QzW+nssEkbqYmxjg0pbHbNHX3mLAEjyyupfdouUO9JLU2eS6dQEIQiFk8tSzQnDnGpSm4dZW6uEWuU0NJ8x0ndia7RtNYT5/FvSPzcOoKeW6Ny3tlMOPYAnx2hcePL6CdR8OuKbTz6fyuZyYLyxuaX9uUXUaSpNZFESaGaXL3iNzUGkIVvDaF+0bmUduQ5PpBWSRNuXZwZ0dUIJwxYwZ+v5+bbrqp+ZgQgmnTptGzZ0/atGnDmWeeyapVq3Z5XTAYZNy4cbRv35727dszbtw4gsHgYS5966MCo9vaeOrEAu4ZlouiKoz5aDvj/7ODR76r4fbhebxxZiHPn9SGb8sjBOI/BD6ZXUaSWqc8p0pN1EJYFv88vS1PndiGh48t4L2N9Tyzso6kSJ0j/eCI+Wt88803PP/88/Tp02eX448//jgzZ85k+vTpfPLJJ+Tl5XHOOecQCoWazxk7diwlJSXMnTuXuXPnUlJSwvjx4w/3JbRKhQ5Bpl1jaU2cJ5YEiJmp8YQtoSRPLKml2K3S02cxcWAOp3bwcVpHHwUuTQ60S1IrZQp4eW09dy2s5a31Ea7+tIJrPtvBitok1VGT10rrZaLt3RwRgbCuro6rrrqKv/71r/j9/ubjQghmzZrF5MmTOfvss+nduzezZs0iHA4zd+5cANasWcPHH3/MY489xtChQxk6dCiPPvooH3zwAaWlpS10Ra2HAng0gUtTEYCuQNIUxE1BVdRkSa3BZxUm9y+s4p2NIf69McylvfycLCfKSFKrVBm1cKrQOUMnw56q16qSSqjf0afjVFPnSD84IgJhU6A77rjjdjm+efNmKioqOPHEE5uPuVwuRo4cycKFCwFYtGgRXq+XYcOGNZ8zfPhwPB5P8znprsCl4tJTuUUNi8aKo+DQFFbWJFhcmSBhCewq6Ipg5pJatsltmCSpVWpKxG1XwamlJsmoCJwa2FU5rLEnrX7W6N/+9jc2bNjAM88886PnKioqAMjLy9vleF5eHuXl5QBUVlaSk5ODovzwxa0oCrm5uVRWVu719x5oa/FIam2qmk5vbx7j+/l5bW0dZ3TKwKYqjCh0URNNUB5KEo/Hms+PA+sqAsSSVS1X6F/oSPoc9uRILz/IazjcVE3nur55LAsKbFj8cWgOc5YHEMk4QoMJA/wkyjdQahotXdR9diCfQ7du3fb6XKsOhKWlpdxzzz28//772Gy2w/q7f+qP9nNKS0sP6PUtoTPQq0GlQ4adB76pJmoIZq8IcMewXDr47bTJcDdPlrGrCl0Lsmjn8rdomX/Okfg57OxILz/Ia2gJAlhXbvL+lloaDEGRR+PPw3LIdGoUuFSKXQIFX0sXc58dys+hVbePFy1aRE1NDcOHDycnJ4ecnBzmzZvH7NmzycnJITs7G4Cqql1bJlVVVeTn5wOQn59PTU0NQvwwOiyEoLq6uvkcKTVWGEzCrV9VsDVsUB0zCCcFd86vxhAqI9t6AJoX08uJMpLUOjXtWg8Ctw6BuMk9C8opcKm0c8mk+XvSqluEZ555JoMGDdrl2KRJk+jSpQvXX389Xbt2paCggE8//ZTBgwcDEIvFmD9/Pvfccw8AQ4cOJRwOs2jRouZxwkWLFhGJRHYZN0xnglTl2RQymhfRN1WXmCmIJAVD2zg5oci50x2lJEmt0c671jdJmKl1v+3SfCf6vWnVgdDv9+8ySxTA7XaTlZVF7969AZgwYQIzZsygW7dudO3alYcffhiPx8P5558PQI8ePTjppJOYMmUKjz32GABTpkzh1FNPPaK6Ow4VAXxYbvLI4lqu7J+NpqayyJiNO9Q7NQWXrgAKQ7JBZhmVpNatabLMzsHQrsmE+T+lVXeN/hLXXXcdEyZM4KabbuKEE05gx44dvPHGG/h8P/SBz549m759+3Leeedx3nnn0bdvX55++ukWLHXr0dSNEjYE/yqt467hudg0BU1JBcG7R+QSM0yeXFJDmZwpKkmt3s671mc5VM7s5OO6IQUIZBjcm1bdItyTd999d5fHiqIwdepUpk6dutfX+P3+Pc46lVLdKNVxix0NJlvDBg2GxSPHFuDQwO/Q2FwX5531IQJxS3atSNIRoGlPwr4n5fN1pcHMJbXE4zF8LqdMlr8XR1wglA4ut00jELcQAjx2la/K46yrr+asjl7+tqoORYGOPh2v3ItQko4YCmAJeLYkNWkGhEyW/xOO+K5R6cAkTJNxff04tNTon1tXuKZ/Fv1z7YzpnUmuQ0VV5ExRSTrS7HHSjEyWv0eyRZjmchwqSyoa+EO/TIq8djRV4YFFVdTETXJdOpMGZXN8oZMeXkt2p0jSEWSPk2ZU2bOzJ7JFmOaKXYIr+vr5ZEuEmqjBQ4uriQvw2jTipuD5FUHcmlwuIUlHCgFsjSrUxC0mDUxNmgG5BvinyBZhmmsaWO89Op9VdSZCQH3cwhICVVFw6wqVUYv2cpKMJLV6Oy+HSliCApfGbcPyEA31dMnPlGuA90K2CCUUoJ1LUOzRqUtYjcdSd42BuIXLprVg6SRJ+qWalkM1dYdWRFM7x2TZLJlV5ifIQCg123XijIJNU7iqr1/uZi1JR4i9TZCpTbRQgY4QsmtUatY0cWZMn0zCSQtQeHNdPccUuhAg7yYlqZXb2wSZbHsLFuoIIFuEUrNil+DSXn6eX1nH31cGeW1tPed2zZBZZSTpCLFzVhn4YYKMNxZo4ZK1brJFKDVTgA5elfF9M0kYFnZdZWF5RGaVkaQjxM6T3yqiVnOS/HWlqb0HmxLs7/ycrNUyEEq7yXGofLIlQigaxeFwAnLtkSQdSZomv6VuXHfafo5dZ5Q2tRZlyjXZNSrtprlrRZNrjyTpv8nuM0qbUq7JYQ/ZIpR209S10ma4j6QzU3afSNJ/iZ9KuZbuwx4yEEo/ogDuSBXd2vqR3aGS9N9BplzbO9k1KkmSlAb2NqNUDnvIFqEkSVJa2NuM0vTuFE2RgVCSJClN7G1GabqTXaOSJElSWpOBUJIkSUprMhBKkiRJaU0GQkmSJCmtKcFgUI6YSpIkSWlLtgglSZKktCYDoSRJkpTWZCCUJEmS0poMhJIkSVJak4FQkiRJSmsyEB6geDzOTTfdROfOnWnbti0XX3wx27Zt+8nXzJgxgxNOOIF27drRpUsXLrroIlauXHmYSpwye/Zs+vfvT0FBAccffzxff/31T57/1Vdfcfzxx1NQUMCAAQN47rnnDlNJ92xfyv/2229zzjnn0KVLF4qLixk9ejTvvffeYSztnu3rZ9Bk/vz55OTkMGLEiENcwp+3r9eQSCS4//776d+/P/n5+fTt25ennnrqMJV2z/b1Gl577TVGjRpFYWEh3bt3Z9y4cVRUVBym0u5q3rx5XHzxxfTq1Qu/389LL730s69ZsWIFZ5xxBm3atKFXr15Mnz4dIVpu8cC+XsOXX37JJZdcQo8ePSgsLGTkyJH8/e9/P6AyyEB4gKZOnco777zDnDlzeO+99wiFQlx00UWYprnX13z11VdceeWVfPDBB7z99tvous5vfvMbAoHAYSnzG2+8wa233soNN9zAF198wdChQ7ngggvYunXrHs/ftGkTF154IUOHDuWLL77g+uuv5+abb+att946LOXd3b6Wf968eRx33HG8+uqrfPHFF5x88sn87ne/+8WB51DY12toEgwG+cMf/sDxxx9/mEq6d/tzDVdccQX/+c9/ePzxx/nmm294/vnn6dOnz2Es9a729RoWLFjA+PHjueSSS5g/fz4vvfQSq1ev5qqrrjrMJU+JRCL07t2bBx54AJfL9bPn19fXc84555Cfn88nn3zCAw88wBNPPMFf//rXw1DaPdvXa1i0aBF9+vThb3/7G/Pnz+fKK69k8uTJvPbaa/tdBrmO8ADU1dXRtWtXZs6cyYUXXghAWVkZ/fr1Y+7cuYwePfoXvU84HKZ9+/a89NJLnH766YeyyACMHj2aPn368Je//KX52ODBgzn77LO58847f3T+nXfeyTvvvMN3333XfOyaa65h9erVfPTRR4e8vLvb1/LvyYknnsiIESO4//77D1Uxf9L+XsPvfvc7+vbtixCCt99+m/nz5x+O4u7Rvl7DJ598wuWXX873339PTk7O4SzqXu3rNTzxxBM8/fTTLF++vPnYiy++yC233PKzPUGHWlFREQ8++CC//e1v93rOnDlzuOuuu1i7dm1z0HnooYd47rnnWLlyJYrSsntR/JJr2JPLL78c0zT3u2UoW4QHYMmSJSSTSU488cTmY8XFxfTo0YOFCxf+4vcJh8NYloXf7z8EpdxVIpFgyZIlu5QZUoFhb2VetGjRj84fPXo033//Pclk8pCVdU/2p/x7Eg6HD8vfe0/29xpmz55NVVUVN91006Eu4s/an2t49913GTRoEDNnzqR3794MHjyYm2++mXA4fDiK/CP7cw3Dhg2joqKC//u//0MIQU1NDW+88QYnn3zy4SjyAVu0aBEjRozYpeU1evRoysvL2bx5cwuW7MCEQqEDqs9yG6YDUFlZiaZpP7q7zcvLo7Ky8he/z6233kq/fv0YOnTowS7ij9TU1GCaJnl5ebsc/6kyV1ZW8qtf/epH5xuGQU1NDW3atDlUxf2R/Sn/7p599lm2b9/ORRdddCiK+LP25xpWrFjB9OnT+eijj9A07XAU8yftzzVs2rSJBQsW4HA4eOGFF6irq+Pmm29mx44dvPDCC4ej2LvYn2sYOnQoc+bMYdy4cUSjUQzD4IQTTmDWrFmHo8gHrLKykrZt2+5yrOn6Kysr6dixYwuU6sC8//77fP7553zwwQf7/R6yRbgH9913H36//yd/vvzyy4Pyu/74xz+yYMEC/v73v7eKL7j/dm+99RZ/+tOfePbZZ2nfvn1LF+cXicfjXHHFFdx7771H5BdVE8uyUBSFZ599lqOOOorRo0fz0EMP8fbbb+/TjWNLWr16Nbfccgs33XQTn332Ga+//joVFRVMnjy5pYuWlhYsWMBVV13F9OnTGTJkyH6/j2wR7sGECROax/z2pri4mG+++QbTNKmpqSE3N7f5uaqqql80o2/q1Km88cYbvPPOO4ftCy4nJwdN06iqqtrleFVVFfn5+Xt8TX5+/h7P13X9sI/17E/5m7z11lv84Q9/4KmnnjosY7F7s6/XsGPHDtasWcOkSZOYNGkSkAoqQghycnJ47bXXftS9d6jtz+dQUFBAYWEhmZmZzce6d+8OpMbWf+7zO9j25xpmzJjB4MGDufbaawHo27cvbreb008/nT/96U8UFRUd8nIfiL3V5abnjiTz58/nwgsvZOrUqVx55ZUH9F6yRbgHOTk5dO/e/Sd/3G43AwcOxGaz8emnnza/dtu2baxZs4Zhw4b95O+45ZZbeP3113n77bebvwwOB7vdzsCBA3cpM8Cnn3661zIPHTp0j+cPGjQIm812yMq6J/tTfoA333yT8ePH8+STT3L22Wcf6mL+pH29hrZt2/L111/z5ZdfNv9cccUVdO7cmS+//PKwdKnvbn8+h+HDh7Njx45dxgTXr18PQLt27Q5dYfdif64hGo3+qOem6bFlWYemoAfR0KFDmT9/PrFYrPnYp59+SmFhIR06dGjBku2befPmccEFF3DLLbcwceLEA34/GQgPQGZmJv/v//0/7rzzTj777DOWLl3K+PHj6dOnzy5jakcffTTPPPNM8+Mbb7yRl19+mWeffRa/309FRQUVFRWHbdLApEmTePnll3nhhRdYs2YNt9xyCzt27GDMmDEAjB8/nvHjxzefP2bMGMrLy7n11ltZs2YNL7zwAi+//DJXX331YSnvgZb/9ddf56qrruLOO+9k5MiRzX/vw7VcZU/25RpsNhu9e/fe5Sc3NxeHw0Hv3r3xer2t/hoAzj//fLKzs5k0aRKrVq1iwYIF3HrrrZx99tk/Gqdrrddw2mmn8d577zFnzpzmMc9bbrmFAQMGtEgwD4fDlJSUUFJSgmVZlJWVUVJS0rz84+677+bXv/518/nnn38+LpeLiRMnsnLlSt5++20ee+wxJk6c2GIzRvf1Gr788ksuuOACxowZwwUXXNBcn6urq/e7DLJr9ABNmzYNTdMYM2YMsViM4447jqeeemqXu8bS0lJqamqaH8+ePRvgRy2TW265halTpx7yMp977rnU1tby0EMPUVFRQa9evXj11Vebx8zKysp2Ob9jx468+uqr/PGPf+S5556jTZs2TJ8+vcVaVvta/ueeew7DMJg6deouf99jjjmGd99997CWvcm+XkNrtK/X4PV6+de//sXNN9/MiSeeiN/v58wzz/zFS14OhX29ht/+9reEw2GeffZZbr/9djIyMjjuuOO46667WqD08P3333PWWWc1P542bRrTpk3jkksuYdasWezYsYONGzc2P5+Zmcmbb77JjTfeyAknnIDf72fSpEktdlML+34NL7/8Mg0NDTzxxBM88cQTzcfbtWvHsmXL9qsMch2hJEmSlNZk16gkSZKU1mQglCRJktKaDISSJElSWpOBUJIkSUprMhBKkiRJaU0GQkmSJCmtyUAoSdI+e+mll/D7/bvsWDBhwgT69evXgqWSpP0jA6EkHQGaAs/efj7++OOWLqIkHbFkZhlJOoLceuutdOrU6UfH+/bte1jLcfHFF3PeeefhcDgO6++VpENBBkJJOoKMHj2ao48+uqWLgaZpctsw6b+G7BqVpP8ifr+fKVOm8NZbbzF8+HDatGnD6NGjKSkpAeBvf/sbgwcPpqCggDPOOINNmzbt8vqvv/6aMWPG0LdvX/Lz8+nRowfXXnvtjxKU72mMUJKOVLJFKElHkPr6+l0SuDfZeV/IhQsX8uGHHzJ27FgURWHGjBlcfPHF3HjjjTz99NNcccUV1NXV8fjjjzNhwgT+7//+r/m1//rXv6ivr+fyyy8nLy+P5cuX8/e//51Vq1bx4YcfttgOBZJ0KMlAKElHkPPOO2+Px3fs2IHT6QRSu50sWrSoeSzR7/czefJk/vznP7N48eLmjXFN02TGjBls2LCBzp07A3DXXXfhdrt3ee+hQ4dy1VVXsWDBgl+04bQkHWlkIJSkI8j06dPp0aPHj47b7fbm/z/22GN3mVAzZMgQAM4666xddodvOr5p06bmQNgUBIUQhEIhkslk88a/S5YskYFQ+q8kA6EkHUEGDx78s5NliouLd3mckZEBQFFR0R6PB4PB5mNlZWX86U9/4qOPPiIUCu1yfn19/f4WW5JaNRkIJem/zN5mc+7tuBCpLUlN0+Tcc8+lurqaKVOm0L17dzweD5Zlcd5552FZ1iErsyS1JBkIJUkCYMWKFaxdu5Ynn3ySSy+9tPn4+vXrW7BUknToyeUTkiQBP7QYm1qITZ544omWKI4kHTayRShJR5D//Oc/bNiw4UfHhwwZQteuXQ/ovbt3706XLl24/fbb2b59O1lZWXz00Uds3779gN5Xklo7GQgl6QjywAMP7PH4gw8+eMCB0Gaz8c9//pNbb72VJ554AlVVOemkk3j99dfp3r37Ab23JLVmSjAYFD9/miRJkiT9d5JjhJIkSVJak4FQkiRJSmsyEEqSJElpTQZCSZIkKa3JQChJkiSlNRkIJUmSpLQmA6EkSZKU1mQglCRJktKaDISSJElSWpOBUJIkSUpr/x+kh8cXku/0egAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.scatterplot(\"email\", \"payments\", \n",
    "                alpha=0.8,\n",
    "                data=data.assign(email=data[\"email\"] + np.random.normal(0, 0.01, size=len(data[\"email\"]))))\n",
    "plt.plot(np.linspace(-0.2, 1.2), model.params[0] + np.linspace(-1, 2) * model.params[1], c=\"C1\")\n",
    "plt.xlabel(\"Email\")\n",
    "plt.ylabel(\"Payments\");"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 단일 treatment 그룹에서 지불금액이 얼마나 큰 차이를 보이는지 알 수 있습니다. 시각적으로 보면 두 그룹 모두 400에서 1000 정도이며, 0 대비 1에서 약간 낮아진 것 같습니다. 이메일의 효과가 5.00 R$ 또는 10.00 R$ 정도일 경우, 모든 변동성 안에서 메일의 효과를 찾는 것은 어렵지 않습니다.\n",
    "\n",
    "다행히 회귀는 이러한 변동성을 낮추는 데 도움이 됩니다. 트릭은 통제변수를 추가하는 것입니다. **변수가 outcome의 좋은 예측 변수인 경우, outcome의 분산의 많은 부분을 설명해 줍니다.** 리스크와 신용 한도가 지불금액의 좋은 예측 변수라면 이메일이 지불금액에 미치는 영향을 쉽게 찾을 수 있도록 통제할 수 있습니다. 회귀가 어떻게 작용하는지 기억한다면, 이것은 직관적인 설명일 것입니다. 회귀모형에 변수를 추가하는 것은 treatment를 검토하는 동안 변수를 일정하게 유지하는 것을 의미합니다. 따라서, 비슷한 리스크 점수와 신용 한도가 나온다면, 반응변수 `payments`(지불금액)의 분산은 더 작아야 합니다. 즉, 리스크 점수와 신용한도가 지불금액을 매우 잘 예측한다면, 유사한 리스크 점수와 신용한도를 가진 고객들도 비슷한 지불금액 수준을 가져야 하므로 변동이 적습니다.\n",
    "\n",
    "![img](./data/img/beyond-conf/y-pred.png)\n",
    "\n",
    "이를 입증하기 위해 두 단계로 회귀를 분해하는 분할 방법을 사용해 보겠습니다. 우선, treatment(이메일)과 outcome(지불금액)에 대해 추가 통제변수인 신용 한도 및 리스크로 회귀합니다. 둘째, 1단계에서 얻은 outcome(지불금액)에 대한 잔차에 treatment(이메일)의 잔차를 회귀합니다(이것은 순전히 교육을 위한 것이며, 실제로는 이러한 번거로움을 겪을 필요는 없습니다)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "model_email = smf.ols('email ~ credit_limit + risk_score', data=data).fit()\n",
    "model_payments = smf.ols('payments ~ credit_limit + risk_score', data=data).fit()\n",
    "\n",
    "residuals = pd.DataFrame(dict(res_payments=model_payments.resid, res_email=model_email.resid))\n",
    "\n",
    "model_treatment = smf.ols('res_payments ~ res_email', data=residuals).fit()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이렇게 하면 outcome의 분산이 줄어듭니다. 신용한도와 리스크 점수를 지불금액에 회귀시키고 잔차를 얻음으로써, 우리는 원래의 반응변수보다 변동성이 훨씬 적은 새로운 반응변수를 만들 수 있습니다. 마지막 모델은 이 모델을 통해 유효한 표준오차 추정량과 함께 `ATE`를 제공합니다.\n",
    "\n",
    "또한 단지 호기심으로, treatment를 예측하는 모형이 outcome의 분산을 낮출 수 없어야 한다는 것도 확인할 수 있습니다. 그 이유는 이메일은 원래 무작위이기 때문에 어떤 것도 예측할 수 없기 때문입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Payments Variance 10807.612416\n",
      "Payments Residual Variance 5652.453558466207\n",
      "Email Variance 0.24991536\n",
      "Email Residual Variance 0.24918421069820032\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td> 1.537e-13</td> <td>    1.063</td> <td> 1.45e-13</td> <td> 1.000</td> <td>   -2.084</td> <td>    2.084</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>res_email</th> <td>    4.4304</td> <td>    2.129</td> <td>    2.080</td> <td> 0.038</td> <td>    0.256</td> <td>    8.605</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Payments Variance\", np.var(data[\"payments\"]))\n",
    "print(\"Payments Residual Variance\", np.var(residuals[\"res_payments\"]))\n",
    "\n",
    "print(\"Email Variance\", np.var(data[\"email\"]))\n",
    "print(\"Email Residual Variance\", np.var(residuals[\"res_email\"]))\n",
    "\n",
    "model_treatment.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음으로 지불금액의 분산이 어떻게 $10807$에서 $5652$로 변했는지 주목해봅시다. 리스크와 신용 한도를 통제할 경우 거의 절반으로 분산이 줄었지만, treatment(이메일)의 분산을 줄이지는 못했습니다. 리스크와 신용한도는 이메일을 전혀 예측하지 않기 못하기 때문에 이 결과는 타당합니다(random의 정의상 어떤 것도 예측하지 않습니다).\n",
    "\n",
    "자, 이제 훨씬 더 합리적인 것을 볼 수 있습니다. 이 새로운 추정치에 따르면 이메일을 수신한 고객이 이메일을 수신하지 않은 고객보다 평균 4.4장도 더 지불할 것으로 예상됩니다. 또한 각 통제그룹 내에서 분산이 어떻게 더 낮은지 시각화할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcUAAAEfCAYAAAAwZN3NAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACVg0lEQVR4nOydeXhTVfrHPzc3SZMuNG2aNi0toGyyqWytoIIsiqLijvobR8VRVHCrg4yIoIKICyNu4MbgPjMogyO4gCAILtiyK+AgiEBLtzQlXZNmu78/0tw2NIUCLbTlfJ7HR3Jzk5zb3JP3nHf5vpLD4VAQCAQCgUCA5lQPQCAQCASCloIwigKBQCAQ1CCMokAgEAgENQijKBAIBAJBDcIoCgQCgUBQgzCKAoFAIBDUIIyiQCAQCAQ1CKMoEAgEAkENwigeJ7t37z7VQ2gSxHW0PNrKtYjraFm0leuA5r0WYRQFAoFAIKhBGEWBQCAQCGoQRlEgEAgEghqEURQIBAKBoAZhFAUCgUAgqEEYRQEKcMAls6EEclwyopeYQCA4XdGe6gEITi1anY6v8/3cvioPp1fBqJV4d6SVS5I1SKd6cAKBQHCSETvF05yqSAu3ryrA6Q3sD51ehdtXFZDjkk/xyAQCgeDkI4ziaU6xW1INYhCnV6HQ6TtFIxIIBIJThzCKpzkJ+oDLtC5GrUSSUewUBQLB6Ycwiqc5kVU23h1pVQ1jMKaYZhA7RYFAcPohEm1Oc7weD5d00pB1QxqFTh9JRpk0g08k2QgEgtMSYRQFSEAHg48OBgCxQxQIBKcvwn0qEAgER0HU8p4+iJ2iAIXARC+s8mGNlEkV7lOBQEXU8p5eiJ3iaU5wwmd8ksPFnx0k/ZMcvs73i5Ww4LThaLtAUct7eiGM4mmOmPCC0xkFwi4KD1bXGkm7VyNqeU8jhFE8zRHF+4LTmRyXHHZR+O89VaqR1EYYRS3vaYQwiqc5DRXv+9AIF6qgzVNY5Qu7KPTX+fcDawt5e0SyqOU9TRBG8TQnsspWb8JPy7Bw35p84UIVtHmskXLYRWFvcwRT0xNIjdayp9RLapRE1g1prLy6PVk3pIkkmzaMyD49zfF6PKSaJDL7mpEkUBSYv62E3AovOx1e0qySmPyCNkuqwce7I62qC9WolZieYeGR7wqxu3xMy7Dw5d5SDFotJS4vMXothVVeJESWdltFGEUBZoPM3C32EDeSUSuxw+6mpymSDsJNJGijSMAlyQFFp/2VfrILXMyrWRQCzMyy8cnoVG5ZcZA7esczM0uUZbR1hPtUQKrBxxvDrfVcqO/sdIiEG0GbJ6joZJAUZmYXqwYRAjHFrEInY7uZmJllE1napwFipyhAAgYm6JjUz4yfWheq3eWrybAThlHQ9gnGFw/3mPj8IEk0mKUdkEcUtBXETvE0JViwnGdsT45LJjnCR+94HS9utvPMhmLsLp/IsBOcVgTji3U9Jm+PSObj3xzq47qIsoy2idgpnkbUlXPzShruX5PHnlIvRq1DjY+IbhmC05W68cW6c0AZYOKpjQ6mZVhUF6ooy2i7CKN4mhBU7qir3zgtw8KS3aVc0jGG7Yc8dGwXRfcoLxAwnCLDTnC6Ea5jTG9/PktGd6ak2seaaztQ7vaKRWMbRhjF04SAckdeSKLAzCwbH4xK4c8rAsfnbLIzf5iVWdmFNTtIkWEnOD2p61UxmDqicfswaLWUV3uFaH4bp9XEFF988UWGDRtGWloanTt35sYbb2Tnzp0h5yiKwuzZsznrrLOwWq1cfvnl/PrrryHnOBwOxo8fT4cOHejQoQPjx4/H4XCcxCs5NTSk3LGtuDrEUE5YU8DYbib1cd0MO9E+R3A6cLge6iXL8tnh8HPLioNCNP80oNUYxe+//56//OUvrFixgqVLl6LVarn66qs5dOiQes7LL7/MvHnzeO6551i9ejUWi4VrrrmG8vJy9Zw777yTn3/+mcWLF7N48WJ+/vln7r777lNxSSeVhpQ7fP7Q85xeBUkKfby/0s9WB3yRp4huGoI2T45LZup6G5l9zUxNT+Dhfmae32hrcLEoaFu0GvfpkiVLQh6/+eabdOjQgZ9++onLLrsMRVF4/fXXeeihh7jqqqsAeP311+natSuLFy9m3Lhx7Nq1i1WrVrF8+XLS09MBmDt3Lpdddhm7d++ma9euJ/26ThbhlDvmDbPyTLYt5DyjVkJRQh9nF7jwKYQU+Ad/GLJuSBPF/YI2RYnLV1OobwuJv9f1jYhyjLZLq9kpHk5FRQV+vx+TyQTA/v37KSwsZPjw4eo5RqORwYMHk5WVBUB2djbR0dFkZGSo55x33nlERUWp57RV6mbWfXxZe965OIUqj49J/S0hKejzh1lDUtCn1xTxH6lOSyBoKyiATqcl0SjzzsUpPHe+BbNBZmaWjQ7t9Op5ohyj7dLoneKuXbvYs2cPl19+uXrshx9+4O9//zulpaVcd911TJgwoVkGGY5HH32UPn36qDu+wsJCACwWS8h5FouF/Px8AIqKijCbzUh1/IOSJJGQkEBRUdFJGvmpI3jVt62sTbhJjdYyqZ+ZdKuBjlEa0gw++iekUOj0odXI/PnrPFXhI1xhsyjuF7QVFAIhgru+ORCqgzrAzAsb7ex1uEmN1jKup4leZr36GpFw07ZotFF8/PHHkSRJNYoHDx7kxhtvJCIiAovFwuOPP47JZOL//u//mm2wQR577DF++uknli9fjiw3/2pt9+7dx3S8JZNnbB9i2HIrvMzMLmbpqHiqS3LZU3PcBLhjU7C7AgbvvZ3167TeGmrGV/A7uz2ek34d4WiN30dDtJVraS3XodXpyI/pxF3f5IeECGZk2ZjUz8wD58QxKMVIp1gduw65mfRdEXaXj7eGmuntz8fbQubA0Wgt30djOJFrOVKorNFGcdu2bUycOFF9vGjRIvx+P99//z3JycncfPPNLFiwoNmN4pQpU1iyZAnLli2jU6dO6vGkpCQAbDYbaWlp6nGbzUZiYiIAiYmJ2O12FEVRd4uKolBcXKyeE45wf8DWGINUAF+lFqPWUW/HFx0VRUT8WRRW+dSUc4B3Rxq5fVUBuRVeFm4v4YsxqUhIVHj8dIzWkGbo1CJWyq3x+2iItnItrek6Drhk1u+tChsiiNZLJBi1XPJpTkiMcf62EsavtZN1Q2fOaAVx9db0fRyN5ryWRscUS0tLMZvN6uOVK1dy4YUXkpycDMCoUaPYs2dPQy9vEv72t7/xn//8h6VLl9KtW7eQ5zp27EhSUhJr1qxRj7lcLtavX6/GENPT06moqCA7O1s9Jzs7m8rKypA4Y1skmGZ+y4qDTMsIjSM+e0Eim2zVIZmlX+QpbHNAjzgdG8cG+sh9dnkKtiovo5fmcOWyXDJEBqqgjVBY5cOvhJdy659oZOKagno1vrf2MIm4ehuk0UbRYrFw4MABIFDrt3HjRoYNG6Y+X11d3fSjq8OkSZP45z//ydtvv43JZKKwsJDCwkIqKiqAQGzw3nvv5eWXX2bp0qXs3LmTCRMmEBUVxfXXXw9A9+7dGTlyJJmZmWRnZ5OdnU1mZiajRo1qMyuohggU7xewp9TLkt2lLLw4hQUjk/n0ijQURWH6+tAOAHd9k8/XuQFDub3Ew4A48Cmo2avB80RquqAtYI2UWbTLwdwhodqn0zMsbCp0ht1BSlLgHK1GFnW7bYhGu0+HDRvGW2+9Rbt27fj+++8BGD16tPr8//73P9q3b9/0I6xhwYIFAGq5RZC//e1vTJkyBYAHH3wQp9PJI488gsPhoH///ixZsoSYmJiQ95k8eTLXXXcdAJdddhnPP/98s427pRAs3k+N1nJt11juWFkr9/bSUCtmg1yvZU4w4/T2VQWsubYDByu8olOAoE2SavAxa5CF/RVeMvuaOTNWx/4yD/O2lXBbT1PYJDMNMD3Dwp+/zlMF9IX6U+un0UZx+vTp7Nmzh2nTpqHX65kxYwYdOnQAAm7K//73v4wdO7bZBtoY1RlJkpgyZYpqJMNhMpl46623mnBkrYNg8f59Z8dR6fHzcL+AK/y9nQ4eWlvApH5mZmYXq+fXrVd0ehW+2F+pupdEBqqgrREsWdpVGcktKw4yc3ASfuC2niZW7CtXk8zMBplxPU10j9Nj1Er8Y/shbu1hQpJgxyEPveMjaR8h5kJrptFG0WKx8NVXX1FaWorRaESvr63ZURSFpUuXkpqa2iyDFJw4qQYf/x6VTL5T4al1BfUSBs6M1asGr+5xqFW+ef9X0SlA0HaRgO5RXh5Lt4R4UqZlWPg+t4J/XZrCoWo/E9bUzp+Xh1p5fqNN1QruHKsjpb3YLbZmjlnRJjY2tt4xo9FInz59mmRAguZBAs5op+WmFTn1EgYm9TNT4gq4jQxaiZ7xeh79vpDcCm+Igcyt8DJ/WwmZfc30T4ygazvRKUDQdlCArEMaJq4JL5y/qaiaOZtDVZ0eXFtAZl8zz2woxulVuGd1AX2FylOrpkGj+MMPPxzXG55//vnHPRhB89KQKHi3OD2mCA1P/WRjY1E1XWK1fDiqPeVuLzF6LbesOBgSb5Ql0GukBo1h3Q4DoqOAoLWQ45L5IT98WUZptR+dLDWYcFP38e5S0XatNdOgUbziiitClF+ORrD2r6SkpEkGJmh6gnHFw2OCRVVexn9jY+4QK45NNmYNstA9yosUBQpeZg2ycPuqAswGmYnnxDPjMPdp3eSCcH0bRQKCoDVQWOXDIEsYtRJmg6zGCmUJ7E4vPeMjws6fw7WCN9mqueVru7jvWykNGsVly5adzHEITgKpBh+vD7dy7+ramMj0DAuGmh+BzHWBLNPuUV51ItfVTLVXw+iloe7Xw0XBw/VtFMLhgtZA+yiZmFINz12QSKVHCVn8PXWehZc2BxaOmevCi+oHa37L3YFENpF40zpp0ChecMEFJ3McgpNAjkumqNLN9IwELEYtUToNdqeHMrefJ86zsK/Mg8vrrbeyDXYjL6xqWBQ8WJLRkItWlG0IWjpeBf72fRFT0xOYlV0csrB74icbCy9Ooazay6R+Zs6M1dMzTke3KC8DElLYXepjt8NNtVdRXysSb1onrbZLhuDYKazyoQAKEvd/W8Aj3xVS7oEZWcXctSqfFzfbyalU2OoI30S4oZ6MdbsFNOYcgaAlElzQ+ZXwi7+dJdUYtDLnpxiZt81OkcunLhi7xcpU1hjPusb0ntVC3KK1cUzZpy6Xi2XLlrF161bKysrw+0M71EqSxGuvvdakAxQ0HUmRMl1Mem5ZEXBv3trDpLqIIDCJx3+TT2ZfM3O32Hl7RDKXp9Qm1ITryXh4SUZjzhEIWiJJNQu6biZ92Nihzw/3rM5nUj8zl3WKIVqrAYILTehljhBekjZAo41ibm4uV155Jfv27SM2NpaysjLi4uJwOBz4/X7MZjNRUVHNOVbBCaAAfiRcPkWduA31SAwev+ubfJZemUqSQVIzSS9Jhqwb0ih0+kgy1i/JqBuDbOgcgaAlIksBhZoD5e569bjBsiSnV0EnS6RF60g0hCaWPdzPLMQt2gCNdp8+8cQTlJSU8PXXX7Np0yYURWHhwoXk5eUxbdo0jEYjn332WXOOVXCcBCfuf36vZNchd4h7M5yrs7c5gqnpCTXSb74QofCv8/2kGXwMjAu4jcIZu6BL6UjnCAQtjb3lfuZtK2FgUiQLt5ew9Mo0FoxMZuHFKQSDCUatxHlWI2fEyKRE+FRNYadXUdur1dVOfXtEMiUun9BGbUU0eqf47bff8pe//IWBAwdy6NAh9XhERAQPP/wwu3btYsqUKfzrX/9qloEKjp9gRujD/cy8t8PBU+dZeOInG+/tdDA9wxKSZTc9w8Ij3xVid/mYnmHBVuURmaSC04JovQa7y0e1z8tTgxL5o8zDg2tDM7UjdRJbbU5idDJ+dBxy1SaWHS5uEaXTcN+afFXtRpRotA4avVOsrKxU+xcGJd7Ky8vV5wcNGnTcBf+C5iWYQBCplTDIEGfQ8M7FKTx5noWecTo+vSKVj0e35+PRqWgIbbAqa0KnsGiVI2irxOgChi+3wo+j2qcaRKidD1UeP45qhYe/K6Tc48dzWLup3Aovc7fYaR+l5ZrPc9lT6lVfLzrKtA4abRSTk5MpKCgAICoqiri4OH755Rf1+ZycHHQ6XdOPUHDCWCNlusRqidZreG1YMsVOP+NW5nHnqnxuWp7HFls1BVUe1uc7qfAqvDTUyoDEQNJAvEFLanStQ6FuJqlCoDnrVgf8r1LLhpLwWasCQUtHASo9CilRWs6K0xOp1YSNt1sjdbz/a6BJt8sL+8vczBgU6jJ9d6QVl6fhjjKClk2j3aeDBw9m9erVTJ48GYAxY8bw2muvodVq8fv9vPHGG4waNarZBio4flINPl4blsw1n+fy4aiUehmnM7Js/OfyVP72fa7qKgqq2+wtdTOup4mZ2cUhmaTBOOXU9YXc0TuemVlCwUbQOqmbLBNUbUowSExLTyCYX//eTgd2l4/YCInbeppYtMvBtuJqPv7NwdyhSbx/SQpajUTnGA1pBh+5rvDqUSLppuXTaKM4YcIE1qxZg8vlwmAw8OSTT7Jv3z6eeeYZIFDs/+yzzzbbQAXHjwTIij+g4ej2h13B5lV6Qwxl5roCPhiVwkNrC3nyPAvTMhIY1SGKXtGB4v4DNXHKzL5mNUsv+FoRdxS0JuqqMN3aw8QXe0u5s088czYXhcQTo3QSv5W4eXGznZeHWgE/j6UnsqGwmoU7HGpPxbRkjShNasU02ij26tWLXr16qY9NJhP//e9/cTgcyLIc0shX0PJoHyUzLT2B5Cht2BVsQZU35HyzQcavwLheJtJidHSM0aoGEWrjlA2VdYjaLEFroW4D7j4JEVycFllPznBGlo2p6Qk4vYraHSPQg7QwpGSj7oJQlCa1Tk5Y0cZkMgmD2MJRgF9KPMzZbOfVLXZeu8gaEgN59SIry/8oU89PjdYy8Zx4bvs6j5lZxVy9LIeNRdWsK1LUeGFd5RqhYCNozQRj7hPOiefjXYc4WBU+HljtU4jRSjw2MIGH+5npZY4gNVqrtpe6tYdJXRCGdIppwCAGY/IiFt+yaPROsbGlFjfffPNxD0bQPATdQ2aDzF1nx/P3jQEdx0qPn2idhte2FpPZL4FNNUo343rWV7qZUdN38Yx2kXQw+FT30NT1NtF4WNCqCcbc71uTz+vDU/guryqsN6VfooHdh9zM3VKrbVq316gkBc5LiZT5Ot9zxE4xoptMy+WYYooNUbfFlDCKLY+ge+i+s+OQFIXRZ8Zyx8qAkRzX08Qdvc3E6mW+GJPK+nwXZ5r0YVfKflDdokHlmh6jUyip9rHm2g6Uu73CTSRodQRj7nf2iqPI6WPhDke9hd68YVZyyt289UugDjH4k7dwewn3nB3HrOxiNMC7I634FNRYIohuMq2NRhvFbdu21Tvm9/vZv38///jHP8jNzeX1119v0sEJmoage8hk0GLQyszMChjECefEh0z8Vy+ykhgls7fUHXalrIGQ7Lmgck0HQ6DvYo4ccBeJBquC1oY1UqaTSY9OArvLpxbhSxJE6wJ6qAWVHib1t4S0jpqWYaFLrJZPr0jF41foGKWhxHX0TjGim0zLpdExxQ4dOtT7r1OnTgwdOpT333+fhIQE3n777eYcq+A4CbqHMtcVkF/pUbPsDs8avf/bAjrE6Pl0T1k9uarpGRbOtwZEGw6PgQRdQYfLwYmYiaC1kGrwoZUgr8LDy0Ot2F0+ntlQzKJdDhKNWi7+9ACbbW7VIAJqLNEUESjUv3JpLhmf5JBbpdAlNnS/cazdZMTcOXUcU5eMIzFq1ChmzZrFnDlzmuotBU1IWbUfs0EmWq/BqJUazBrNKfdwbddYluwuZVJNMoEpQibBILHrkIdrvsypFwMJ5wqaut6G57xE7vqm/vliByloaUhAt3Yy24qr+eqPUv49OgW9RqbS4+fnYhdmg9zgnNlQ4Ay59+/6Jp9Pr0jlms9r637fGG5FKwWMncSRu8mIeOOppcmM4h9//IHb7W6qtxM0ITkumf3lTiaeE8+U7wuZlmHB5fWHdZFG6TTM/LaAdy5JweNTeG1rMdd3jaVrXAR3fpMfNgYSzhU0tpuJuxo4X8RMBC0RWYJ+iQb+9T8H+RV+MtflhbhJFUUJO2cqwxhKGT9ZN6Sx0+Flh93N1B9tah1j0Lg1VLJxQMQbTymNNooN6ZqWlpby448/8uabb3L55Zc32cAETUdhlQ+NJDGjpgHqkt2lPHFeAnOHWEPiI68PtyKh8M7FKdidHqq88OjABLYXu/mpzmo4SDAGEnQF1X1e1oj6RUHrosLjIy1K5uWLkrn+i9x6btIn0hPqJeDMHWJlziZbyPsYtRIxei2ypDBuZe3CMDVay/YSD5E6A52iAwX+wZh8XZUbEW88tTTaKF5xxRUhWaZBFEVBlmWuvvpqnn/++SYdnKBpsEbKHKyqVbK5pGMMY78MJNtk9jVj0Er0jNfz6PeFqqL/qxdZ+df/SujYN4FKX6AbeUOyVeFcQRckGxslcxVSzxUpEnQEpwYF2OHw80dpNQZZCmuUkqK02JxePh+TRqXXj8enEB+hYcpAC/d9G5p8c8uKg0zPsPBEegLlXoV2OglJknjiJ9tRXaLhFplCIu7k0WijuGzZsnrHJEnCZDKRlpZGu3btmnRggqYj1eDDHlvbTfzw2EhXk54/rwh119z/bUDmzVHt55wEA49+X1Bvlfz2iGTV5RPOFXQ0mSsROxG0FHZVapm45gAP9zODFH4BGKPXMG39IeyuYuYOsRIXITFtfRF/7Z/AwotT2FlSjc+PWrd49+qA6s2s7GKmpScwZ3Nxo1yiQiLu1NJoo3jBBRc05zgEzUxBpUftnRiplbggOYKH+iWwtzQQBzYbZG7tYVLrr97b6ajpCqDhyZ9s3NE7noXbA2nqsgbOTzaSEedXjVfd8ozgavZoMleiVkvQUjhYEVCxaaeTMOo06lwJ1vJ2jtWzx+HmvrPjePRHG5nrCvj3Ze2Zkp6Ao1rBIEt89Uc5l3SM4baeJiAwh4KC4n4aH044UrxR0Pw0WaKNoOWyq1LLuJUHMBtkZp9voWOMlpToeB79PtDhIrfczcRz4us1G86v8BCrj2BUpxiW7C5lbLeA0bwwJZKBJt9R3Z7hDGVdROxE0FJoHx3QBNZqJP72fRFmg8wT6QmYDNqQuPvcIVZSo7XkVngpcfmxVXmY/IMNo1bi5aFWnt9oU0MQ0zMsSErobvPw3Wc7vRYF7zHPHUHz0aBRPPvss8PGEI+EJEls3br1RMckaGKCq2BrpExvs4Fip49IrcScC5OY9F0ht/eKY1Z2fVm3qekJFDl9LNrl4I7e8czfVoLd5ePKTlFN4vYUsRNBS8GkU3hrRDJef+D+z63wUuZReCo7tC4xc10BmX3NzN1ip6DKQ6d2evW5B9cGnntmQ3HIHILArnHGIAvT14cuPLOKXBREyQxJlMROsIXQoFE8//zz6xnFrVu38uuvv9KjRw86d+4MwO+//64eO/fcc5t1sILjo320li6xWsb3iefKpbV1g/OHWXnhwkQOVSsNCiDvL/OortNxPU2cGavDpFO4Zb0tRO5q6nobPUanHJPbU8ROBC2FnAofKH7+KPM2GHuHmnILDUzPsHBWnI6qOs8HX1P3cXJkoEm33eWjvyWCSf3M+AFFgXk1i8y6msKCU0+DRvFwybbPP/+cL7/8kqVLl3LhhReGPLd27Vpuu+02pk6d2jyjrOGHH37g1VdfZdu2beTn5zNv3jz+9Kc/qc8risKzzz7Le++9h8PhoH///syZM4cePXqo5zgcDiZPnszy5csBuPTSS3n++ecxmUzNOvZTSfcob9g08wlrAsk0dqcn7I7tnIQIHlpbiN3lI7Ovmb6JBmJkhWI3jO8TH5JJNy3DQkn1sbk9RexE0FJoHyXj8MDCHcVqQhmEd3n2jI/gke8KGdfLRFeTnqcHWwLGUoJIWQo5d1+Zh1mDLQxI0JFX6WNmdnG9z66rKSw49TRa5u2ZZ55h/Pjx9QwiwNChQ7nrrruYNWtWkw7ucCorK+nZsyfPPvssRqOx3vMvv/wy8+bN47nnnmP16tVYLBauueYaysvL1XPuvPNOfv75ZxYvXszixYv5+eefufvuu5t13KcaCaj2hm8uvK24mmp/YOVbV9btzeHJ5FZ4uK2niYf7mTFFSOw6VM2avGpyK32qQQy+z8wsGwbtsYeog7GTgXGB/wuDKDgVeBXQHKZ72k4XiCHWnRdzh1h55LvAQrGfJYLCKi+zsouZlV3MC5vsmCO1DEiMwKiVePaCRCK0EntKPZR7JdpHhZd2q9UUFrQEGv0rtnfv3iPupkwmE3/88UdTjKlBLrnkEi655BKgftcORVF4/fXXeeihh7jqqquAwG63a9euLF68mHHjxrFr1y5WrVrF8uXLSU9PB2Du3Llcdtll7N69m65duzbr+E8llsjwzYV9frDGaLE7vXx6RSolLj9FVR4MMkz5oXYnOH+4Fa9fYUZWIQ/3M4c1sOVuL0Sd7CsTCE6cfKefKq+iZp0+s6FYNWzB5sKKAqXVXuwuH9MyLPzvkJsZWaFlFg98W8CyMWnsL63GXq0wKzvw/JxNdt4daeXfo5K5aUV+SFyxp0krQgYtiEbvFM844ww++ugjKioq6j1XXl7ORx99RKdOnZpybMfE/v37KSwsZPjw4eoxo9HI4MGDycrKAiA7O5vo6GgyMjLUc8477zyioqLUc9oiCgGh41cOay48LcPCx7852Fvq4cmsYvKrvMzfVkxSlI7bVoZKtE1YXcCBstrmq6KxsKAtYdDKjP8mn3nbSnh6kIX/XJ7KO5ek4PL4eePnQ8zKLmbuFjsdY/Vk9jUzf1sJFZ7wsfi1B6uwROrreVNuX1XAGe20ZN2QxrIxqXw5Jo0xHQ0iyaaF0eid4tSpU7ntttsYOHAgN910E2eccQYQ2EEuWrQIm83Gu+++21zjPCqFhYUAWCyWkOMWi4X8/HwAioqKMJvNIQlEkiSRkJBAUVFRg++9e/fuYzre0nDHpnD7ylKeP9/Cx6NTyS504vMHesHd0TueJbtLyexrZm+ph8czkrC7wnceD9Zcvbezfr+5t4aa8RX8zm6P5+RfYA2t5ftoDG3lWlrDdWh1OgqNndQM7SidzC0rDobEyxduL+GRARYmrSskt8KrvrYh78uGovCyiH/Yy0mpOoi15lgVsOckXGOQ1vB9NJYTuZYjeQWPSebtk08+4YknnuCll14Kea5Pnz689tprjBgx4rgH2ZIJ9wdsTe7WDSXg9DoocPp579ciHh2YQLVPYWw3E0t2l3Jt19gQA/fG8GS6xGrZUxo6+fskRKg1WvO3lTCpn5l0q4GOUZqaBJlOp0y2rTV9H0ejrVxLa7mOAy6ZyEo/XWK1PDowIUTdyWyQcXr8zBychF4DhjrOkPd2OlR3a10DOn9bCbf1NIU1mGeYY+jQ/tT8TVrL99EYmvNajikzYvjw4QwfPpzCwkJycnIASEtLIykpqVkGdywEx2Cz2UhLS1OP22w2EhMTAUhMTMRut6MoirpbVBSF4uJi9Zy2SLsIreru3FHi5qG1hfxjZApzt+ST2ddcr6/iPavz+XBUCresqK1DfGWolcd/KGTiOfFqKnnveB0XmBWkmppCIdsmaI2UuHwcqlZ47sIk/md3hwh4H96Iu26Bvt3lI1In8f6oFH4uDpV4W7TLwdsjktVOMaLcqPVwXIo2SUlJLcIQ1qVjx44kJSWxZs0a+vXrB4DL5WL9+vXMmDEDgPT0dCoqKsjOzlbjitnZ2VRWVobEGdsaCoqq5h90e079sZCXh1rJqfCEdfPsLXWrkm7nJEQACqYImRlZNj67Mg2rgeOWbRMi4IKWhEGnZfw3B3jyvAQ6mwIawWaDzAsXJrGzpJqH+5l5b6eD3AovD64tYMkVafyYX8U5CRE8u6GYgipfPUWou8+Op0usljXXdqDc7SXJKCNLsLHk2O55MVdOPg0axWCrqPPPPz/k8dEInt8cVFRUsHfvXgD8fj+5ubn8/PPPxMXFkZaWxr333suLL75I165d6dKlC3PmzCEqKorrr78egO7duzNy5EgyMzNVF3BmZiajRo1qM26Fw1GA/zk8fPhrCTMHJ9FOJ/H11WmUeRS0kp+z4qKYs8lez83TPc7AryUuSt2KWqu48OIUbv7qIJVeP7KkYWMJIRO1MbJtYjcpaGmUVwdi6B1j9Bwod/PsBYlUeRTuWJlXzy2aW+Hl52InM7MC2alBdZtIncTU9EBYold8BIVOL2sPOnkyq5hPRyeT51T4Id+JX4FFuxzMGmQ56j0v5sqpoUGjGGwVVVBQgF6vb7B1VJCgS7KkpKRZBgqwZcsWrrzySvXx7NmzmT17NjfffDOvv/46Dz74IE6nk0ceeUQt3l+yZAkxMTHqaxYsWMDkyZO57rrrALjsssvadMurHJfM01kBjdM7Vubxn8tT2FPmYcLqgIpMl1gtLw+18uDa0NY3D6/LZ+bgJB75rjaxoNITaEy8udBFiVNHhcfHvd8WqhPcqA9f9qGTteS4FFINPiECLmhxBOUG95e56WE2sMPuqldqMbNGsq3aq9AhRs/U9ATe2+ngzFgdmX3NvLDRrs6TGYMszN5QzMKLUzAbZPZVKGSuyw2ZX41RgBKel1NDg0Yx2CpKr9eHPD6VXHjhhTgcjgaflySJKVOmMGXKlAbPMZlMvPXWW80wupbJ/go/Y7uZmFmj+O9XNExYXTvR9pR6eX6jjU+vSKPK4ydSp+GVLcXsKfWys6SaCefUap5G6zTqijkoTzW+TzyvbivBo5jJq6yql5U6LcNCVkEVT2YV8+5IK0lG0XxY0LJINfh4c3gye0rdOD0eOsTowt6j7aN0TFhTGyOcO8RKtdfHMxtqVWqMNb1JM/uaqfL6mXNhEo//WFjPwGb2NR/1nheel1NDg0bx8FZRonVU6yRar0HW1CTQnB1HVqEzbJuoH/Kr6BKrZ5ejmv87y8SoTtF0iNEF5Kx6mugUqyPJqGF/We3k9gNP/GRj4cUp3LEyj4f7mVm065CqiaoogbKPsd1M6ip3zbUdhAi4oEUhAUmRGhKMEfgVDRFaKew9urfUHWLcMtcVsPjyVJ4/30KpJyDzlhQpU+lRmLvFHtb1GnytrOGo93xjBPOF56XpaXTxfkNkZ2ezcuVKKisrm2I8giYmKQL6Wgx0idXSJVZPtFZi8gAzQYlGWYLHBpoZZI2oUfYv5pYVeTz6g409pR4y+5kZlhpJidNLodPP29sPMeGcQBKBogQmobNGQu69nYFuGnO32NVi5zt6x/P+rw6gVvXm3ZGhIgIiK09wqkk2anD7oaDKyy/FrrDybu/sdIS8xulV+KnAiQ9Umbd2epm8Sg8P9zMzNT0Bs0FmZpaNW3uY1NcZtRLnJxuPes8HBfOPNFeOtJsUHB+Nzj594YUXyMrKYvHixeqxm2++mRUrVgCQnJzMV199RYcOHZp+lILjJjnCR1mkhinpFiQJjDqJSo/CnM21K9npGRbcPtS+cRCYWNPX25jUz0xBZRUzs4tDCpmfvSCJh9YWBmqv2gUy9oL1i5l9zXSO1bGvzBOyQg6ucgfEIUTABS2KNIOPveUKqdE6ip0SpgiJ/1yRiq3KR3KUFoM2oItal2ChvsUY+Bk1G2RyK7xqPLLuLjGqjmF7e0RySIPuhjiSYH4wjqjVhhcQEJ6X46fRO8XFixfTvXt39fFXX33F8uXLefDBB1mwYAFut7tNJ6y0ViTA6ZW4b00B+0rddI6NUFPHobZ3oqzRhF1xdmynI1onqY9nZtkY283EzhI3dpePl4daqfZ6mVYjKJ5b4WXuFjsVXj9dTToMMjw2MIFpGQn894pUdVILEXBBS6LQLVPkVLjm8xwe/aGIvWUeJMDjV9hQ6OSx7wMlTHV3bW8MT2ZdbjlJkVqmpifwwoVJ/GN7YFE4NT2Bh/uZ1ZZrfRMNfHtte7JuSOPylMbLuoWbK8E4YsYnOfx5RV49Mf9j9bwoBAQMNpQEDK1y1Fe0bRq9U8zLywspW1i6dCmdO3fmiSeeAAIKAx9++GHTj1BwQihASXXAxfLqtkO8PNQa1vgFM0sPX3HuL/PQPlqnKtkE4yHnJxtZflUHyjw+TBEyX+4tDoklvrrFzoejUnjivETG1ylgfntEMqmRgTowkSUnaCkUuCQeXFuA2SDzyAAzlR6F+9fkM7abCVkDmf0sfLnXwaLLUtlQFJBJnJlVxOQBFg6UVTMru5jnL7BwR+/4eolmZ7bT8sC3+Xw4qj1pBu8J3/N144i5FV7mhVWXahwiUac+jd4pSpKEz1e7+li7dm2IrFtKSgo2m61pRyc4YXZValFA3cVpNOHFvA9WeNTdXvDYtAwL7+x0kLmuQI2JGLUSg61GPF4vq3Iq+anAxZf7K7n/XDPrcst5b6cDWYKp6YnkVPp46qeikF3pXd/k83VuNemf5PB1vv+0X5UKWgY2Z2DBd2sPE8VVPv5Rowv88W8OfH7YWuzi2m5xPL+xiJlZxTyzIZCh/eDaAiyROqamJ3BGrL6eOtTMLBsajcSeUi9f7K9sknu+bhwxNVrLrT1M+IFo3bEZRAga2NCwye2rCshxnb7i/o3eKXbp0oUvvviCv/zlL6xatYqCggIuvvhi9fmDBw+26Ua9rZWDFV5+L3Xz0lArD60t4ECZmxmDLExfX7uafeUiK89tsOHywaR+Zjqb9ORWhMYDJSlgEF+9yEpptYecSn9IXPLloVYez0igsCogE3ekzLtgR3ORJSdoKSRHBWpsJSnQ9HdsN5MqmF935zd3iJV95baQ+7nM7UdT49cM54Wpcgd0VX1+muSeD2almg1yiAzdi5uPfZfXmLKP041G7xTvv/9+vv32Wzp27MjNN9/MWWedxUUXXaQ+v3btWs4+++zmGKPgBGgfraXSq1Ba7eWJ9ARMEVoiZIlJNdlxk/qZidNLzB+ezMtDrZyfEkm528sbPx8KSZDpbY4gs6+ZZzfYSIqKUI0qBCbRg2sLOFSNahCDx8Nl3ik1c1BkyQlaDIrCy0OtyFIgIztKK6n1vYeXYRx+P+8v8zAzq5hf7NVhvTC/l3qYPMDC1/vLcXoVdpf6jjl2VzfupwD/HpXMuJ71x3esu7yggT18zKdzG7hG7xSvueYa4uLi+Prrr2nXrh133nkn2ppO64cOHcJsNnPjjTc220AFx0f3KC9/xOtpp5f5Kd/JPavzQ+oUI3USPgU2FVWHaDfOGGTh5S0lakPVuso2f5S5w64ufUr4/nJyzdKr7s4x+FhkyQlaAkUuP2/9UsLU9ARcPgWzQct3eVVHvZ+nZ1iYt62E1GgtkXJgJxnM4q57v9tdPjL7mtlR4maTrZpbvrY3elfXUNwv3Wo44V1esOwj6EIVJVLHKAh+0UUXhewOg8TFxYkkmxaKBJwdpyXf6eeseL0anH9mQzGp0VpeuDAJRaJeRur09QHh7+8OVrFkd6lqRGUJ0qJ1IUk5qdFaxvU0oZUkpmcksHCHI2SXOax9JEOubo8PDfetySe3wismn6DFoACJkTI7Stxc8/lBvrqqPR6fn3MSDGGTz3rFR/Dvy9qDBJPWBfq4TjgnnqdqVKMm9TPTxaQn57AQhKxBNZLHEj5oqEC/KYQwjlT2cbpyzMX7v//+O++99x5z5sxh//79ALjdbnJycnC73U0+QMGJkxLhw6doMMiS6ioJtsW5Y2UevxRXh19xVnnpl2Rg4jlmtJqA8s0Lm+wUVHl56rxAUk5qtJaJ58QzZ7Odm746yAub7Ew8J57U6ECM5q0RyfSNDaSUZ8T5WTI6hZVXB1LTT+cMN0HLQN2FrTjI68MDJRdVXnD5FF7abKtXxD9/uJVJ3xXyu8NNO50GgwwvXJiE0+vn4X5mAGZmF3PP6kBj87qLwz4JEfXi640JHzQU93N5m0YIQ5RIhdLonaLf7yczM5MPPvhAFf8eOHAgHTt2xO12M3jwYCZPnsz999/fnOMVHCft9KDXaHjugkRsVT76mPUkGHX84+IUkiO1LNrlqNdUOFYvc+NXufVcQeO/yeepjAQ+Hp1KhcevdhOA2rrH9y9JYYutmqd+KkJfpyNAB0PQtSN2iIJTT91dmE4j8cnoVFw+hYJKN7f2jOf5jTa1hdq5CRHkV3qwu3z0SojA4fLw6EBLvW4aS3aXcknHGLrE6pk12MI7Ow7xl97xFFR6Qz67sbu6huTe4iNkzokVQhhNTaN3in//+9/58MMPmTp1KitXrkRRar+g6OhoxowZw+eff94sgxScGDkumb1lbqp9PtrpNfxrl4M9ZV4uX3qA//vqIKM/O8CjAy10iQ2skYxaqWbVDP+6NIXRHSPJ7GvGWSNwbDbIJEbpcPv87CwJv8vcWlytpq2f7inegpZLcBeWGq1FL0tE6TRYIzVoNTLPb7SpdYrnJEQwd3MxhVV+pmdYKKryEG/Us788IOmWGq3F6VVYuL2EO2ukDm/66iBPZxfzt4EW/rO7lL99X8S4nibg2HZ1R5J7E7u8pqfRO8WPPvqIW265hb/+9a9h20P17NlTlXwTtCzynX7aR0VQ4fFz7+pcMvua62Wt3f9toHnq93lVaICCCi/3ri5gxiALf+ph4s5VtWUW0zMsmA0SO+1uBlmNYVexddZMp32Kt6DlYo2U6RKr5Y7e8dy6Io8XLrDQx2xUk2WCHTCMWonPrkilzOPn91I31qgI7luTx55Sb4gXZWw3Uz25xAe+LWBqegIbf7SRbjWw8ur2x7SrE3G/k8sxKdr079+/weeNRiMVFRVNMihB06EAhVV+7l59kIf7mUPqBOvi9CqUVgdWrX5Qzwnqnx7uHp2ekcCUH2xckBzBG8Ot3LO6IMRozttWu3ASWaaClkqqwcfLFyVz/Re59IrXc1a8gYIqLw/3M7NiXzmXdIwJZGlrJYqrffxlZfga3GA7qGBHmro4vQrWyECMvWOUpiaxpuG50FB/RBF6ODk02igmJiZy4MCBBp/funUraWlpTTIoQdOR45K5e3WOOlGNWonIBlrj1Ogas2iXg0n9Laq0m/+w93R6FSo8gdd+n19NQZWNNdd24GCFl18dbiJ1kiqeHNCIFFmmgpaJRKBBeq94PeP7xHPl0sBc6RKrZfIAS0jz7YCHRFblDoOG8JkNxWqpRkMZq7F6DZ9ekUphlQ8JWW24fbjhE7Jrp55GxxTHjBnDwoUL+f3339VjUk1DvpUrV/Lvf/+bq6++uskHKDgx6mauvbfTwdwhVnQS9USEp2dY0EgBg3hH73jmbAoU3Ru1Ur2bxKiV6GuJYGp6Qk1dF5S7vXSLlQMtdDbaVVHkSf3MDEjQiQktaLF0iNIwqX+CagAhoGhT93HQQ1K3cD/odYHaUo1nN9jqza2Xh1qJi9Bwzee5XPzZQdI/yeG/uX6u/TJPfRyUfxOya6eeRhvFRx99lNTUVIYMGcJdd92FJEm8+OKLjBw5khtvvJHevXvz8MMPN+dYBcdBXcWK3AovC7aX0LGdnnnbatX8M/uambethOzCalXFI5hgMGOQhbPi9fV6y+VVenlvp4MXN9uZPMBMO72Wgiof/70ilU4xgQmskeCCFCMpEWKXKGi5pBl89UIKDYUYpDqru2DsPOgNiYvQMKZzO86K0/HhqBQWjExmanoCFoOmntLThDUFjO1mUh8HDZ/oj3jqabT7tF27dnz99dfMmzeP//73vxgMBn766SfOOOMMHn30UR544AEMBpFJ0dJINfh4a0Sy2qliR4mbKL0Gu8unJhFAYGJHyBLOGlWaKK1ERrKRcrePt385pHbAGGQ18uRPRewocZPZ18z7vzqo9ChctOSA6u55bZiV2dk2NQnhaO6fhmIoAsHJIkanCev2PPyxps6/5w6xUub28sGoFEx6iWKXn9k1rtS6r1l+VVpIuRPUN7BOr0JJtQ+/FH4cIiZ/8jim4n2DwcBf//pXvvvuO/Ly8igoKGD9+vVMnjwZg8HA3r17m2ucghMgRieR2dfMgpHJTOpnZo+jOqz7VF/T9smolciwGnlyfSEbCqs5LyWKsxMiiNVLePwKBTWrWUmCW3uY6qnh3NfAKjgcdXvDHe5KEghOBjsqAq2dXr2otuxh0S6HWswPgTnx1HkWYnQSC0Ym896oFJ7OtjH5ext/XpGHRpLJKXerohbB13xwSTIGWWJaRiDUkBpdW/ZUN0PbqJUwaLVMXJNfr1vN2yOSRUz+JHJMMm8N8csvvzB37lyWLl1KcXHx0V8gOGnkuGSqfT7mbrHTK17PowMTcFT7qfQEBMLLvQqKAvO2lTBzUCJvbS9iWoaFAxXV3NozPiTRIKCBWsC0DAtl1V58fij3htc7PXwV3FBJRkMSVqJ7huBkcbDCy55SLynRMrMHW0gwatl1yE2FxxfSI/SVrYFM06npCZzRTheiTFPs8qIAvcx6nh5kweH2M6y9EVuVN8SLMi3DwsLtJUxNtzArO9BqL+hNKa8OjGN+TWgj+LmpUY1vSiw4cY5qFH/99VcWLlzIH3/8gclk4qqrruLKK68EAhmns2bN4ptvvkGn03HTTTc1+4AFx0ZhlY99ZW7mXJhIlVfhzytC1Tfe23lI1SJtp5fUljnPXJDEbStCjdXMLBtT0xN4aG0Bk/qZidFriNGFdzMdvgpuyP0jWtcITjXBtlEGjYaeZgNXLcvBbJB5aaiVR747GNaFGqXThBz7pbgao07Dg98WMD0jkfNTdFS4lXpJMzOzApnaMVqFuUOsROs1JEYEpBhzXbLa97RufeRNXdIQrtOTxxGN4oYNGxgzZgwul0s9tmTJEmbNmgXAtGnTiImJITMzk7vvvpvExMTmHa3gmEmKlKn0KvQy67jhy4OqesetPUw4vX4WjEzhl2InEVoND64txO7y8cZwK64GdoDto3SYDTJ+4ImfbPzn8hTmDbMycU3tjvIfFyezv8zD1PQEZAn6J+gbdP80JGElYiiCk4XVoDB/mJXcSi8ef+C+v7WHiUe/D3hFDu+n6Pb7ySkL6DwH+5G++XOJGmefkVXE04OTqPD4w86hfeUexq0MdKsZ19NEL7Men0lLmsHHv0cls9Hmxg9HnTuC5uGIRvH5558nIiKCDz/8kEGDBrF//37uu+8+Zs+ejdPp5IEHHiAzM5OYmJiTNV7BMSJL0M8S6I4RNIh1G5MG44lnRMm8PDSJQ9V+UqN1bC5yhjVWe0vdjOtpwlfTULXSQ407KIH2UTpsVR4KKn3MyCoOqbOiAQeQaF0jONVYdD66xmrxKBqKnF612XA4V6Y1UsPBykDl7jsXp1BQ5eGQ08slHWPYWFRMrF7ijt7xjFuZx8P9zGHn0A67u16D4OB9b5Al/rXLwdhuJhQJDDoZ6lUKC5qTIybabNy4kTvvvJMRI0YQGRlJjx49ePrppykvL+fee+9l+vTpwiC2cGxOHzF6LX6gS6w2RNF/QGKgcXCV1087vZa/fV/InavyuWJpDgoSz16QWK8U452dDs6M1fP+r46a5AANV5zRjiqPwoQ1+TjcClN/LGp0nVVdCSvRPUNwKpCAIqdCYZWHMreP1y4KNBuu68qclV3M3C12IrQakiK1fLy7jAlr8pEU6BqnZ1CykQUjk+mXaGTh9kBrqPd2OuolzbwxPDCHbu0RvkHwToeHO2q0U2dmFXP157l8kaeIxLOTyBF3iqWlpXTp0iXkWPDxBRdc0HyjEjQJCpBTpfCbo4oko4bJA2oV/cMpdtSVrXriJ5tawyhJgdWTw+XF7vJxsKZTwLQMC5lr85kzJAmnFx7uZ+aMdrpjjhEKCSvBqSY2QkOxS+G5DcUsGJmC3emt1zB4eoaFu1blY3f5eHmole9yy+ljMVBU5eOPMicLdziwu3zMGpyI2+un3KsgozA1PYFuJj1nxcpoJbC76tdFQuCxxajl/m9D45B3fZMvEs9OIkfcKSqKgiyHrvCDjyMiIppvVIImIcclM/6bfAyyRLlHOapix8w6ih1Or0K1LyCI/OJmO0adhgU7DjF3iBW9Bj4YlcKS3aW4fHDIFWgfNSu7mAPlHnVlHMSolYjRa9lQEhiTWPUKWhIKoJfBpJeYlpHInlIPk3+w8XS2LaSUaV7NgtHpDcyl23rGc/0XuYxbmccLm+xMOCces0Fm6o9FVHoVZmUX82RWMbIk0csk08HgIzkiEC4I7kTrYtRKROs1onj/FHPU7NMVK1aQl5enPq6qqkKSJJYsWcLWrVtDzpUkiQceeKDJByk4PgqrfJgNMtF6DRaD9pgUO4JNUWcNsnB+SiTbbE7GdjPxdLZNzVbN7GsmRi8xYU2tcV24w8H0DItau2jUSswfZuWWFQcbXcwvEJwsFOCLPIU3fi7kjt7xLNxewtPnW0NcpzMGWZiZHVpq5vQq5FZ66i0qg1qo/jrnzciykZGUSkpEbbigd3wknWN1IUL67460kmgQxfunmqMaxf/85z/85z//qXf8/fffr3dMGMWWhTUykN326PdFfHVVaqMUO2plq5KxVXpIidZR5vERqdfw/ubQruFRWom0mFB3aW6Fl3nbSvh4dCo/21wMTY3k9q8Pqooeog5R0JLIccnc9U0OU9MTeCbbxv19zRg0CgsvTqbE6eVMk0Hd1R0+V9Ki9apofjCj+8xYHdMzEoiUa5d8Tq/C3jIvCRE6tedh+wgfKe019K1pBxWj1+LyeLEaA8X6d32TH2IsTyjxzO8Hvw/JUw3OSvD5kPw+8Pkg+H+fF6nu4zr/D3e83usbOO9Yzg13Xrjj3kEjoWvDHZtOlCMaxW3btjXbB59KFixYwCuvvEJhYSFnnXUWs2fPZvDgwad6WE1OqsFHj/gIzAYZvUYTsoNbtMvBy0OtITHF14dbKXX5avotFnH32fHM3liE3eVj7hAr950dx2s/H+LWHgFd1PPbR7KhoKreD4bd5SO7wEnveB1erzdE4ir447G7rLZbgNgxCpoVRWnwx95R4iOxvIizXD4mJ1UjHSxlX76PxAgJg8vLO6vsXNUxkkXt4e1txfi9XiI1fm7sEo3r2+28pvGypbKCgZERLF/mYIvXh1Hyc1EHI88UlFPp8mKQ/GR8E0mk5EOvD/38Ln4fURVefsqrQPL5sOPnIouOPI0fj8dHBD4i3jkxwyQpgX3ruaf2W2gyfN3Pbtb3lxwOx2kV4lmyZAnjx4/n73//O+eddx4LFizgn//8Jz/99NMxtb7avXs3Xbt2bcaRnjgKsNGhYd1BJz3NETzyXSEvXJjEXoebM016nB4vydER5Fd66BSjp8rr4Yqlta5yo1ZianoCj/9ow6iV+HBUCrscnpA08qfOsxChlXj0+6Jad+lwK33i9RxyeomJ0Kqu03DlIE3lSm0N30djaZZrqdktNGplfiI7gzrHiwsLSYiPO6adSWN3C8e8gxG0GTzDr2L74Cuabb43icxba2LevHn83//9H7fddhsAL7zwAt988w0LFy7kiSeeOMWja1pyXDJbbU66x+lJMMrYXT6226uZlV2sGqiJ3+aoBmreMKvqDoLa5qjBfytI9dLIw2WpdozWcuHiAyExxVnZNrUDxymRdFMUUPzH/cN8RAPQhK4h/D46lZZiiDTWP897vGP1Iiknf+3b/qR/ouC0oJkXOaeVUXS73WzdupX7778/5Pjw4cPJyso6RaNqPgqrfCz6rYwXLkzi99Jq5l2UxAGHizjJw52dI3n1+z+I8viIVXxoXX6e/7KIKX1iqXZ7kRUfOsXHmUXlnFdaQJTGT/s/bJxfVIBO8SErPrSKH63io+svkSzbXUqk5Oe27tEsX+7gL1Ue9fnig7DyzCgqNnsYWVnNzqIqXDWfoVX8mN41EKE9sZ1Bz2oXOo2m/ut93la3W4g71QMQtDkUWYsiaZC0MshaFI0Msgx1/6/RoMjaeseV4GOtVj2uHPY6ZG3teSH/r3nPw44rdV5X73Pk0OcOH6sSHQslpc32tzqtjKLdbsfn82GxWEKOWywWioqKmv4DnZXovv28jlvK2+SuoSO9fojPzzqvBxb7OF/xo6mJLTwOsKbm/4fzQ/1D3wf/sQmWh3vNTrgu+O9fISPcOb8G/tcDGHL4c3mHHzh2RIGQ4EgokuaIP9YeScaNTDWB8w5UKXjQ4JVkvJKMX9aQ2s7Ango/neKM7HB4UTQy51qjMBl0lHglviuoxkXgM9LbR7PR5uaQV8KHhvM7xLDqYDVeSYMkywxJa8fXB13c1tOErNPy6vZyqhQNGlnLxHPNnJugRToeYyPLIGnqGDBtwMDQtkIMwiieYnbv3n1Mx4Poykro/e/Xm2NIjUKu+U8gAFA0MopGU/Nf4MdTkWUUqc6xmv8rGqnmsaw+F3x97TmHv0YTeC+5zjGp5v+yDJIU2LHUjIE6z1PzeWHfV5IC4zz8szQ17xkytrrjldTXIIUvydbqdOyQU9hVpjBnsx2zQWbGIAt3rMyvd+60jATObKfj1yovlkgd7fQSnx1y0yVOz0ubi5k4NgETsMNezdidDnINgTCEUSvh7Wvmhbr9S/snMN9XzDtlEt9cFsf4AX6KPRIJOoXIKhu7PJ76g/XX/OdVCJRnHLv342i/Wa2JE7mWIy0OTiujaDabkWUZm80Wctxmsx1RzDzcH7Axqy7JYT++gQqahfq7hZof0sNcRrXumprnNDJoD3cZ1VmZh3MNNXQ8eCysyyjw//wiG9b27WvO0zbCtVVnnIePKbhrkDSE9PM6RiQaUq9tmNawMzngkrnrkxwe7meuiaHLdIzRhS3B6BUfEShzWpmPUSvx2ZVpAUGLoUk81C+BbcXVxEVo6Banx+7yqa97eaiVt34pCXmvYIjX6VWwS5FckKjU+ftGNcu1tobvo7E057WcsFFUFAWn00lkZGRTjKdZ0ev1nHvuuaxZs4arr75aPb5mzRrGjBnT5J+nHKYG1BLwSxrcyHglDX6NjKwLxBpknRafRku+S6lxGdW6jjqaDOwu95PcLoLcKgVrTASKLBMdoWVLiZdyn4RXkhnWsR0rDzpxKTK+mtcjy/RPjmJtvlt9T42sYXBqO77Jc/Fw/wTiIqSw8QlFI1Ps01HikYiL1GExEPLjrxoFWWbf/gN06tKl5tjhxkNT+96tAMfu3VjayI9XS6du67IusVrG94nn7m/ywnbHmLOpmMfSLWoyWonLxwsXJlLk9HNfnS4xMwZZQnqVPr/Rxs3dTWwsqg6RU4SAgcwucFHl1glBixZCo43i559/zubNm5k+fbp67NVXX2X27Nm4XC4uvfRSFixY0OKN48SJE7n77rvp378/GRkZLFy4kIKCAsaNG9f0H6Y34L507FF3EfV2CxoZRRsujlDn9Y18zx8dWvR6LfetLWJyRjK/l3l4cbO93ir4g1EpaDUSN39Vv39csCzj3YuTcfng8ho9yC6xWp67IIk/Stx4/QqGeD13fVU/QPjvy1J4bmVtMfK0DAv3bCshN8XL6Iz2DAyTWaIAX+f763XPaOiHw13uQjEnNcGXJjidCLYue2+ng1cusvKn5YH7P9gdQ9ZAz/gIpv1YyF96x6PXKDxwbjxv/VLCH2Vu+ica+b/lOSEZ1dPX1yrbBOnYTseSy9sTqdNw35qAa3ZaegJnxOrJr/Qwdb2NHqNThKBFC6DRRvGll16iW7du6uOtW7fyxBNPcP7559O1a1c++OADXn75ZaZMmdIsA20qrr32WkpKSnjhhRcoLCykR48efPzxx3To0KHpPyzCgPvmCU3/vsdAR63M1hIf156VwL5yD4t2OeoJHT91noW9Djfd4/W8PtzKvatDn3tla0mNfqmGe2saD6dGa7mjdzy31Gla/PJQK11itSHF+katRHKklpXXdGBtbhWVNT84Qam4huSrclwyt6/KOzXlG4LThmDrsqnrbWp7NSCk0e/U9AT2lHqZkWXj8zFpvPVLIa8PT6bY5eO7vKojyiVCYA7sL/OQ3jmSNIOPdy9OYXuJJ2QOTsuwUFItGmu3BBptFH///Xeuv/569fEnn3xCfHw8ixcvJiIiAq1Wy5IlS1q8UQS48847ufPOO0/1ME4KFV6JXYeqkTUBXdIHzo2nzO1V6wqNWqmm+N6m7v4+uTyVSrcfRYI5G4uxu3xMz7CgqaOXGq71zYNrC/j0ilSu+Tw3pLPAn1bkYXf51HrFoEE8knxVXbdWkKN12xAIjgdLpJanByexs6S6QenDoBJTQZWXpwcn4fQqTPuxiBu7m8K+Jlon8djABGQNnJNgIFr2k1aj3qTVSKpBhFrd1DXXdgC8CE4tjQ6yuFyuENfo6tWrGTFihNoto0+fPhw8eLDpRyg4IXIrvHz5RzkZViN2l49XtpZwRqyeuVvszMoupsqjqGo0EGisesMXuWwrrmbc13nc2TueJ9ITmLethEPVflXZvyFBca9fIeuGNJaNSa3XWWDCmgI+HNW+UX0Tg26tutTuLAWCpiHHJbPmoJNf7NUs3OHg9eHWkP6H0zIsfL2/nAnnBHoc/ml5HuNW5mGv9vH68GQ6RMu8NSI55DWvDbNyZjud2hPxzysOUuqunSvl1d6wc6fcLQxiS6DRRrF9+/Zs2bIFCOwa//e//zF8+HD1+ZKSEgwGsYRvabSP1nJZpxiezS5i3jArdpePzLWFTK9pfnqkbhlOr0LmugK6xukxyBBnkNXXQfjWN1kFLn495MEgKczMLlbVcYLvW+72MjAOVRi5IYJurbqfdcLCyALBYRRW+fArINf0OTyjnY5J/czMGGThg1EpLNxewiUdY+p5RSasLmDdQScl1Qrlbh9PpCeoyk6zs21st7sbbLQtFnwtm0a7T2+88UZmz55Nfn4+//vf/4iLi+PSSy9Vn9+8eXO9hsSCU0/3KC+/x+t5Z6cPWZKYnpHAGe10WAwSX4zpQEm1l+kZCSzc4VAN2OEp42VuhRmDE/n7Rhv7ygOC4e10Ur3Y5BvDk4mQ4Y8yD51NkapbKeh6kjUQo9ei4D1qll2wxU5WTReBJKOsup8EgqbCGimzNqec+841Mz3DQkGlT20TFbxve5n1argB4L2dgbniB2Zk2ZjUz4xPISSxxn/Y59R1/QcXfIcnkYkFX8ug0Ubx4Ycfprq6mq+//prU1FQee+wxYmNjATh06BA//vgjEyac2qQSQX0koEOMjnE9TdyzOh+zQWbmoAR+L5N4cO2BkNjfvG0l2F2+einje0vdAAxJjeH7DcXq5E+N1rLw4hS226sZZDXy0Np8tWdi+ygr/x6VzF+/L+KO3rUi4HM22RstAi4R2FEGYojiB0PQ9KQafMwYlMj2EjcyCvEGWV3M5VZ4ef9XB4+nW5i7xR6SFLNwewmKEjB2fmpLQFOjtYzraaJjOx1T0xNUA2rUBpRtFPxiwdfCabT7VJZlHn/8cdatW8fnn38e0mopLi6O3bt3k5mZ2SyDFJwYlR4fZ8XrcXoVbu1hwqiV1ZZRUNsI9fXhyXw8OpUv95aqE3lahoWv9pXT0xzBmbGBiZ4aHVhL2V0+fimu5sXNdrIKnCE9E8evLuCMdlo+HNU+rAh40JUkEJxKJMCnSPzt+yIm/2BDQgkJEYzraQqbFPPsBUm8/6sDo1ZCA0TrJAYkRvB4uoU5m+3cuSqfFzfbmXBOPF1itYHWa2vy1fs+uOBrTChBcHJp9E7xyiuvZNKkSQwdOjTs8+vWreOFF15g2bJlTTY4wYmjAEZZUlfAsXqJKJ2Gh/uZgVpXkNOrUFjl5f5vC/h4dAoTPVDh8VPu9nFrTxN3rMyrt1Ke1N/CnE02XhpqZWZWqEpQ0F2EEj5mKbJIBS2FUrcfp1dhQGIEjmofPeO0/OuyFLw+CQUl7P27s8StZmWf0U5LcqREr/hEbvwqt54BXXhxCvmVHvaUesV93wpo9E7x+++/P6JodnFxMT/8EEZNWnBKyXHJFLv83L8mnzeGJxMboWXsl7nMyi5WV7IDEiOYlp5AlE7D4+kJVHgU7liZx12r8pn6o41Kj4LZEFjhBif604OTSI7ScHN3EylRsiprFSSYONBUSQUKAUmuDSWBazqtmoAKmpXkKJkusVpmDk6i2g8uH/xa4uHPXx9ka3F12Pu3W5yeT69IQ1IUHv+xiH3lCr8UuxowoNV0iNHRJVYrkmlaAU2me3Xw4EGioppHs09w/Oyv8HOo2s+eUi/FTm89V9DC7SXc2TueOZvt3PzVQZ7OLmZfmTfECM7IsnFrD5P6nk6vQrVPYeKaQmZmF5O5toC5Q0IzRWcMspBX6UUjwRvDTyyLNKhuk/FJDhd/dpD0T3L4Ot8vDKOgSegZ5WVKuoVrP8/B5vSiIDGjxuX/3k4HT50XmnE9PcNCtVfhlS3FlHkUbuxu4vdSN73M+rAGNCPJyB6Hm9eHJ6v3vVjktVyO6D794osv+PLLL9XH7777Lt9++2298xwOB2vXrqV///5NPkDBiRGt11DlC7g9vUp9V9DYbvVjJjOyQmWqDlfo6BKrJSVKy/g+cVT7Aj8cMXqJZVemkVfp5UCFh0W7SvF2jeWaLwPJPZP6mell1tPTpD3mpAKhbiNoTnJcMvetycNskInRy+woqQ5RtnllawlPD7LQIUaPD4V9Djdv/2Ln1p7xamw+2KR7zoWJTPquKEQz9Z7V+QHxiuFW2ht1JEf4aiQMa0MSjU0+EzQ/RzSKu3bt4rPPPgNAkiQ2bdrEtm3bQs6RJInIyEjOP/98Zs+e3XwjFRwXyQbQazS8e3EycQZtPfUNWdNwnWIQo1aijzmCqekJxOgk0mJ0jFmWE5K5mlPm4c5V+WT2NfP+rw5euDBJjUPmVniZmV0cqGO8Ie2YJ75QtxE0J8H7656z47hvTQEP9zPTJVbL2G4mdR7IEtz6da0u8GMDE+olq01cU8CSK9rz3qgUyqr9dIjR8dLm2lrdCasLmNTPzPkpRrHIa8Ec0Sg+/PDDPPzww0Agw/TVV1/lhhtuOCkDE5w4CrDN7mFvuZez4vRstTnrqf+fk2AIK1OlqfPv14ZZefzHQrXcYnqGBbNBVhN0ZmTZeHqQBadXwaCVmHBOPDvrrLaDHK8hC8YlDx9jQ7qpAsGxYI2Ua7wfOpxehRX7ypk8wBKyC3x7ZHLI/RdO9MJskNlf5qunafqz3a3OFT/wQ74zpH43aHiF9mnLoNExxUOHDgmD2MoIuB0LSIvR4fZBqTsQQ8zsG1DseG9UCgbZz5vDQ2WqnjrPQowu0Dnjg1EpzM62hZRbhIsxxhu0dInV0s2kZ2aWDb8SXvHmeBINhLqNoLlQAJ0GXhqaTFGVB6NW4pKOMfV2gb8dcmPUSqRGa3lsYEAAY3pGbXkSNFy+EZwrQVEMvxIIQQSl44JJb7mViogttgCOq59iRUUFDocDRan/FaalpZ3woARNQ9AtlGDQstXmZNEuB/f3NaP4FRKMWvaXulGQeOuX2jY55yZEMHtDsdr7beHFKSFdLyC8e3VvqZsXhyZT5vapCQqH70qP15CJYmdBcxBM4NpeUk2fhAhQYHqGhSqvv94ucOEOB28MTyav0qsm4RwuenFGrL7BUETdPooGGV65KJnrvsjFbJDV3eJvDjd9zZG0jxCLvVNJo42iy+Xiueee44MPPqCkpKTB8470nODkEnQ75ld6eG3bIZ45PxG708djPwYm9bT0BOZsLsbpVdSkGqNWIrOvmY1FAfdnpcd/VPfq/OFWDrl8yBIcLA+stnMrvCE96S7vGEX3qKPLuzWEULcRNDXBBK6H+5mRJaj0Kbyz4xALRqbUu+ftLh9RutqsVKj1mrxzSQo/26rVnebhc+XClEg0wPwa4/nuSCs6/JgNMhPOiQ9ZOHaO1ZHSXiTcnEoabRT/+te/8q9//YvLL7+cQYMGYTKZmnFYgqYg1eDjjeFW8iu92F0+bFVeHl9fO6n9HDnJxqiVKK328cbwZPY43PgJJBwkRcpUuv3889L2ROkkdpVUU+pRyCpwkhQp8+wFiTz6fRG5FV7mbgnIup2IQRQImoO6CVxzNxfzWHoiczbZeWVLMR+PTiW70IlfgUW7HDw60MKuEnfY+VLh9iNJgVjU4f1IZwyyoNco3NQlkiEpEaqXI9clM65n/fZr96wuoMOYNBIiAvNXzJmTT6ON4rJly7j11lt56aWXmnE4gqZEAgYm6MiP1jJ/uJXiqvotaxrqHxec0PEGDUVOL3M212o/zhqcyCvbDvHAOXH4kJiRVRziTjJqOaESDIHgZBD0pLy308EjA8wkGeG9i5P5vczL2C9re4K+MTyZTQUVnN8+Jux82VvqYe4WO9MzLJRX+/h4dCqFVV5sTi+JRi3T1xcx/6IkBsZB0MuRavDRyxze3boqt4oXNzdeI1jQtDQ60UaSJM4555zmHIugGUiO8OH2SyRESPSuyTQN8t5OR71WUK8Pt9IvUc+/L0shI8nA3lIv09eHrman/ljErT1MpLXTh3Un9YgzcEPnSEZZJaHrKGixBBO47C4f5W4/87cdwhKprXdP37M6n9FnxhKpg5eGhiZ8vTzUSqxeYlI/M5E6iZe3lmDUShRWeil3K8zMKmL0mbGUVIe6/CWgp0kbNhktKDQuNIJPDY3eKY4ePZpvv/2WcePGNed4BE2MBMTqwCPryCqoYnqGRZ30dpePSJ3E1PQEnF4FRQGDLPHSZjv3nZuA0+ujYztdgy7WSk/9hASnVyGnwoPDBWnJTSaYJBA0OXUTuPaU+RiQZOT3Uk+9e7pXvB63DzYUVROtlWqyTnXkVnjobdZzsEKDw+3H7vQya3AiT64v5Pv8arXkwun1o5Xrt0xLC9NCqm6HmmMpYVIIxEgLq3xYI2Xhej0BjimmeMcdd/DAAw9w6623kpqaiizXX8VYLJYmHaDgxDHp4OdDXko9Cu/tPERmXzNnxurYX+bhhY32kD6K71+SwuSBiTxc0wZqekZCg4k2KVH1xQCMWomCKi/3ZxeLYmRBiyeYwFXl07K3zEu0ThNyT6dGa7mzd7wq9B00XE/9VMS9Z8dT4vLzs71ajT0+lm5hX7mP1GgtD5wbzxM/1bZMWzAymXPjteRV1hquoFHeX+knu8DF/G0lIfOxMbW4wSxaoZDTNDTaKA4cOBCAX375hQ8//LDB80T2acsjOcJHcZSWnSXV2F0+ntlQTGp0oE4qKOQdjAdmrisM6am4cIcjZHcZlLMqqvTyTHYRr1xk5YFva1e6bw5PRtbAOxenUO7xQZhVrljVCloaLo+XtBgtU74vYP5wKxNqkmUaqj3M7GsmOUrHDV+GGstnsm2M62kiQiupBjH4ujtX5TM9I4EpP9hCDFcHg480A1S5dSHzsbElTEIGsWlptFGcPHkykiR+ulorcXo/veL1zBhkYfp6G7kVXhZuL+GDUSmUu/3sLfUwr84qdWYd/dN520rU5AGLUeZAmZtKn8LMwVZe2RI4Ly5C4qx4A5l1Gg3PG2alZ0zoalWsagUtkUSjTJnXz5T0RPaXVfPFmDT2l3swajVhQwSyBjR1VG3qGsve5gicYWodnV4Fi1Gr/ruu4TqRWlwhg9i0NNooTpkypTnHIWhGclwyB50KsRESiT4tk/qZ8RNojOr0Kuwt8zAzuzjkNXVLM+wuHzpNIMvuyZ9squEEGNUpltJqL70TjFz7eU49Lcie13bgrKja88WqVtASUYAdh7zct6YAs0Fm4jnxzMiy8Xh6+PBBepKRnXZXyHs4vQqdY3XsL3PTqwH5xCidJuT8nQ4vaVYJieOvxRUyiE3LcWVCuFwu8vLycLvdTT0eQTNQ4vJR5fFhkLXcszqfmdnFvLfTQblb4UCZhxFpUXSJDV0f1S3NmJZhYVeJi2c2FIfEO6JkiYMVHjq001NWHX61erAyVA3nSKtageBUke+E+9YE3KS39jAxI8uG2SDTM15fry3a3CFWtBo/FZ765U37yjwgSczOLqr3uukZFvaXukPO32F3n3CGqZBBbFqOSebthx9+YObMmWzcuBG/38+nn37K0KFDsdvtjBs3joceeojhw4c311gFx4lBp8VT7WVrca0Q8eFKGi8PtfL8Rpvq+pw7xEpptZfMvmYWbi/h2QuS1NWoUSvx7AWJVHsVVRGnoYSc9lFaoNYwilWtoCVSUcfdKUmoajP/tzxPbX12Zqyeg5Uens628fchSXSL04fMiWAc3u7ykdnXjClCUr0yGqBjOy3Tfww0aq97/pCUiBNycwoZxKal0Ubxu+++49prr6VLly7cddddvPHGG+pzZrMZgPfff18YxRZIebWX/WVuOsYEJvGtPeoraTy4toBPr0ij2OXFKEvM3VzMkNQYZA08e4GVj34t4aurOvBNbiU+P5S7/czKLlbfI1xCzvxhASWbuqSGSUMXq1rBqSbeIKvtojrH6vj7hUlM/bEQpze09VlmX3NA8k0rUeb28eGoFDbbqlEUQjJHZQ3sKfXgUwJG1qdAklHDzd1N+AFFgSW7SxnX04TLJ5Hj0pxQwpmQQWw6Gm0Un3nmGc4++2xWrFhBaWlpiFEEOP/88/noo4+afICCEycpUqaTW8/+cjfTMiwNJgHkVgRihp1iZO7oHc8z2Tbu7BWHV1H4v7Pi8Cl+0pMMZK4t4MbuppD3yK3wMm9bCcvGpFFa7aN9lDastJtY1QpaIlVuL48OtHD/t/VrBoOGLphgMz3DQpXHj4yC1w8v1qg9BTFqJQYmGpmRVcTGomr12E1d0ugdr+P2VaFxS5Fw1rJodExx69at3HjjjWi12rBZqMnJyRQVFTXp4ARNgywBKPROMLBwewm94iPCKmnsLfVgd/mYPCCRZ7JtgY4aksS4r/O44cuDXLE0l/xKH68PT2Z4qrHee9hdPhIjoGs7mfJqL7kuOWwrnOCqdmAcQvFG0CKI1GtVgwj12z5BTbPthAjmbSvhtpX5xEZoeWFTcT1VqJeGWnl+YxHXdY0lNVob4g0JLgg/uCSlnnKOULBpGTR6p6jT6fB4PA0+f/DgQWJiYppkUIKmxeb0YdQFxL3/0juex38s5NkLEimu8qki373i9UTIEh+PTuWQy8fYbiaKq3yq5inUulkn9TOTFqNThb/rrnT/KPNy04p8sfoVtBoUoKiyvi5wcGcItYkyew+51Z3j1uJq/to/gQNlbpZckcrBCi+p0Voe/zGwQ9xks7HkilTa6TX0rOM16WDwUVgVXoxflFGcehq9U8zIyOCzzz4L+1xFRQUfffQRF1xwQZMNTNA0KEBulcKGgmruWJnPvG0lPHhuPO30GuZsDjQ4fWGTnTKPwqTvCrlyaQ57St3Imoa7aPiBB9cWUO72k9nXzLSMBNZc24He8TrVIAbPFatfQUsnxyXzs706rPdkcHIk/xiZzD8vTUFSFBIitUxNT6BLrBafH3Y73DyZVcz+Mi/PbrDxQ56TSzoGNgdOr8IvxS5yyr18ma9woI7nJJhwdvjnHU8TbkHT0mijOGXKFH7++WeuvfZavvrqKwB+/vlnFi5cyNChQzl06BCPPPJIsw1UcHzkuGTu+iY/xMAlR+tUxQ5qjj/wbQFju5kA+PKPci5MiUSuaY5al7qCxcE+jDOziil3e8mrFOUWgtZHYZWPhTsczBqcWK+E4q5Vedz3bQEl1X4W7DjEnavyeXGznckDLKzLLadXvB6zQSZzXQHzh6ewqbCKPgkRTE1P4IULLJgitIxbmceflueR8UkOX+f7URBlFC2ZRhvFfv36sXjxYnJzc7nvvvsAmD59On/9618B+OSTT+jZs2fzjBJ49913ueKKK+jQoQMmk4n9+/fXO8fhcDB+/Hg6dOhAhw4dGD9+PA6HI+ScHTt2MHr0aKxWKz169OC5555DUcJFvtoGdesCg5mn24qrGxT5To3Wcm3XWCaszsNslOvFS546z8L7vzpU4xg8rtXIxESEV/0Xq19BS8YaKWN3+XB7/Sy9Mo0FI5OZ1M+sKjw5vQr3raldNAbDCA/1S+Bv3xfWiH4rfJdXxdVdYpn2YyGzsospcwfOC+c5qZtwtvLq9mTdkCbCDC2EY6pTvOCCC8jOzuaXX37h999/x+/3c8YZZ3Duuec2uwRcVVUVw4cPZ/To0Tz22GNhz7nzzjvJzc1l8eLFADzwwAPcfffdLFq0CICysjKuueYaBg8ezOrVq9m9ezcTJ04kMjKS+++/v1nHf6oIumlW7Cvn5aFWPH4fnWMNTMtIwK8E2kflVnhVI1e3XOOFjXbuOzuO9y9JAST2lFbzytZAHdb0DAvztpWoK+o/f52HQYb5w6xMWCPKLQSth1SDj7dHJLPL4Sa3wo0sacIqPJ0Zq2NqegIQmDc7S9zsKfUi1XhUfP5AWCEoj9hQ+CEYNxRlFC2TYzKKQfr06UOfPn2aeixHZMKECQBs2bIl7PO7du1i1apVLF++nPT0dADmzp3LZZddxu7du+natSuffPIJTqeT119/HaPRSM+ePfntt9+YP38+9913X5vUdlUn/CE3u+yV9LREcU2NHFsw7Xzh9hKmZViYmWULKbXIrfDy6I82AJaNSaXKo3BbTxPROomMpAi6DrGyrdgVopk6K9vGmms7UO72inILQatAAi5PkegWF8UfpV72lLrDCkzsL/Mwq6ZecXqGBUlR1I4xwfKNuvKIwdcJoYrWxTEbxX379rFv3z4cDkdYt+M111zTJAM7VrKzs4mOjiYjI0M9dt555xEVFUVWVhZdu3YlOzubQYMGYTQa1XNGjBjBrFmz2L9/P506dToFI29eJCA1UqKdPoJIbSRXLgvVJ52ZZePTK9IoqfbywoVJVPvCT2SLQaZ/koFIrYb4CAmjRmHNQWe9FfWeUi/lbm9Il3GBoKUjAdU+MEXIvLPjkLpIDC4eg54RqG2mPT0jgZeHWrEYNdz/bWGIxwVqm3gfLmjRFJ4T0Wmm+Wi0Udy3bx8TJ05k/fr1AGENoiRJp8woFhUVYTabQ3Z7kiSRkJCg1k8WFRWRkpIS8rpg/8eioqIGjeLu3buP6XhLwxifilcXzb6y+g1Ug0X7939bwAejUpj0XWG9ifzyUCu3rDioSsBNz7BwZjuZaK0U1oC28zvZUeSn2C2RoFeIrLLhPUI5T1PRWr6PxtBWrqW1XIdWpyPP0AmvonBH73gWbi8hs68ZWQMZSUbuWZ0fIoTv9CokRWp5JtvGzd1NqkEMyiVCoG43PkIT0sR7VraNbvpo9KV5JzTW7Zpkxq+1q3P0raFmevvzjzrPWsv30RhO5Fq6du3a4HONNooTJ05k48aNZGZm0r9/f9q1a3fcAwry9NNPM2fOnCOes2zZMi688MIT/qwTIdwfMOiSbQ0ccMms2ueks0kf1ohF6QLtcfwKGGRCJvIFKZHcvyaPPaW1qh4zsmxM6mcmUifVW1F/cEky+9xKiIzb2yM6cnknqVlXsq3p+zgabeVaWtN1HHDJJLih2Olj4fYSxnYzBeTZ/LDd7lL7HAYxaiX+KPVw/7lmFBQWjEwmLkKDNVLmtWEpePx+2uk0/GlFXogxBajSRdHrBP4uB1wy4z8J9fiMX2sn64bOnHGEXWhr+j6ORnNeS6ON4qZNm3jooYeatIXUvffey9ixY494TmpqaqPeKzExEbvdjqIo6m5RURSKi4tJTExUz7HZbCGvCz4OntMWKazyUeVTeG1rMa9eZA2Rsnp5qJW/bwrESX4prmbKQAuv/1xCQZWPcT1NVHn8PD04if1lbso8ipqY4wfObKdnb5mbVy6y0k6vwWLQkBAhkXHYhL3rm3w+vSKVjDi/cPEIWhwK8D+Hj1K3F1OEzLSMRO5ZXStA8ewFiWof0rpu0C4mHT6/gs3p49ava8+fMcjC6DQDfoWwxvREY4qif2Lz0mij2L59e2JjY5v0w81msyomfqKkp6dTUVFBdna2GlfMzs6msrJSfZyens6TTz6Jy+XCYAjcPWvWrCE5OZmOHTs2yThaIu0itKzNKebBvmZSo7V8MCqFal9AyWbOpmJ2lLhDFP6npicgS1KIC3VahoVFuw4x4ZyAaylWJ5FglNleorCvzMOiXQ5mDbLg84VvyvpDvpMUY6TomShoceS4ZBR8PLfRzjPnJ2LSB4S/JSkg3P3CRjsAn4xOxeb0UVDlobDSy94yD+ckBIyf2SCr5RvT19sYYEnj3NjmEb8XnWaal0bXKT7wwAN88MEHVFRUNOd4GqSwsJCff/6ZPXv2AIFs059//plDhw4B0L17d0aOHElmZibZ2dlkZ2eTmZnJqFGj1G329ddfj9FoZMKECezcuZOlS5fy0ksvMWHChDaZeQqBVbDTp3D/uQnERmj5Ls/Jbw4PT6wvRAFGdYohs69ZFT52ehV6myPq6TLOzLIxtlugXGPu0CTMkVquWJrDzKxiXtxs547e8Uxdb2uwVtHnRxTxC1okJS4fWg08lp6IVpKIN2qZuyWg9hTsIWp3+ZA1ErOyi1CQeCo7IFrx5xUHOVStcN/Zcer7Ob0KFV5/s9UiisL/5qXRO8XbbrsNv99Pv379GDNmDCkpKchyaFG2JEk88MADTT5IgIULF/Lcc8+pj4Nu13nz5vGnP/0JgAULFjB58mSuu+46AC677DKef/559TWxsbF8+umnTJo0iWHDhmEymZg4caIqRtDWUIAv8hTu+ibgzlwwMplSt8KiXYcCMRPCK/xrJUld+QYxG2T6JETwcD8zkVqZ2dmF9YxmZl8zLq+Xt0ckc9c3+SG7zIXbS7ipSwpiJStoSQRlEOver+9dnMy8YVYmrgl0sxjX00T3OD1+xc+dveJ46rAFY+a6AhZeXJvAZ9RKdIzSAL7jrkU8Unap6DTTvEgOh6NRci6//PILN998MwcPHmz4zSSJkpKSJhtcS6Y1BK0PuOSQ+N6/L2vP4z8WMr5PPE/8FOgs/ni6hcx1BfUM2F96xzHlh0C8NTVaW6/NzeFtdQCmZSRww5mRpBl8ZB3S8EO+E58fPv4t4FptTsWO1vB9NJa2ci2t4ToOnyNQo9xUI25hMmhD5sfcIVaezrbVS57592XtmfRdIXaX74RF8BXg63x/Pbfric6f1vB9NJYWkWiTmZlJeXk5L730UpNlnwqal8MD8nM2FTN5gAWHy8v0jASskTr0MiHxk6ChO7Odni6xWvaUehnX0xTWnRpU7oDAD8n5yUZ1xZoR5yfFGEmh08dNXVLESlbQIgmXtGI2yKS10wNwx8q8ervC6RkJ6oIRAvf+Dns1j6dbGJCop7Oxfh/RYyHHJXP7qtDPvX1VAVk3pImY/Emg0UZxx44dTJkyhdtuu605xyNoQg4PyG8sqibtj1Ju6RHPdV/kkhqt5eWhSczdUt+FGqPXsPDiFL476KR7fMRR2+q8PSI5JLtUSFgJWgOHz5GgV+SOlXk83M8c9r7vEKNTF4x1vSZ2l4+sG9JOePEnsktPLY1OtOnUqVObFs5ui4QLyP+pRxzbil0YtRK5FV52H3LXE/2enmFhe7ELtx8skVp+rjm/LsGGq1PTE5jUz0xfs/a4fgwUAi6sDSXBLECB4ORx+Bw53CsS7r7/X4mbt0ekMGOQhYUXpyATkD80G+QmSSYTbaVOLY3eKT766KNMnTqVq6++uk2XL7QlggH5ZVemUezy8UeZm0qPwpd/lDN3iJXMdQWUexXe23koxIU6b1sJt/U0UeLy8eDaQLJBONmr/EqvWrc4JKU97SNCP/9oUlS1sZM80ZRYcEoIzpEvx6RR4PSh19TuGt/b6ahX1xvcFfZNNKDTSKp7NTgnUiJPvCwiaKibupRD0DgabRTXrVtHbGws6enpDBkyhPbt24fNPj2aQo3g5CIBsgZsTg8gUeb2cl3XWOZsCsQEz06IwO7yqbFBQBU5thi1NTJwXuZvK1ENZ29zBI/UJBVk9jUzd4u9Xo1UYwyeiJ0IWgISkBAB1X4ZBUV1p+ZWeEkwaJjUz4yf2pi73eUjWqepF2efkWXjfGsafuXEtEhFdumppdFGceHCheq/V61aFfYcYRRbJkkRUNFOR6lbIVoncfNXeWqD4NRobT2t0+kZFlKjtVT7fCE/EM9sCCjfZPY1q9l3soawq9jGGDwROxG0FFINPvaUB+7nuvNht8NNZB0DGJwfek34tlC/l3mZlV10wtnWIiZ/6mi0UQwWyQtaH8kRPn4tBatRotAZOplzK7zM21bCh6NS8PghSqchRifx8pZibu8Vx/zhViasrq3XOiNWT36lh9RoLXaXj8s7RtE9qn62XWMMnlDmELQUJCA5Uss3OVW8/kttOMHnh3YRkrpb1BDQBq7y+cPeu3tL3YztZhIej1ZMo41iXU1RQesh6MYs8/gxRUQQIdefzHaXj01F1SHlFR+MSkEjwbytJTyRnkCcUctDa2tjHDMGWehviaC82kuuXN9d1BiDJ2IngpaEy+Olj8VQL5zQJVbLzMFJ7Cyppld8BAfK3HTR6NQC/8PjjeP7xJHZ18zuMh8Soq1Ta6PRRrFXr16MHTuWsWPH0rNnz+Yck6AJCboxl16ZhscPL2228eGoFLYWV+NXYNEuB3f0jmf+tlrRBadXwe2DPYfcbCyq5uouqAYx+Pz09YFOGTNrmq7+e1Qy1igduRVe2kdr6R7lParBE7ETQUsi0SjjKPXw1ohkxtdRuLmjdzyPfBfolzg1PYEusXp+L3WTGKnlnUtSqHD7KajyqvHGnvF6/rxCJI+1VhptFPv168cbb7zBK6+8Qq9evbjpppu4/vrrSUpKas7xCU6QoBuzsMqLTgOjz4zlljoT9s3hyazNKQtR6Ai0k5LQaAJ1W2nRurCuUJ0cmOZmg8xOh5ebVtT+kMwfZuXq1KMbPBE7EbQEFOBAhZdIvY6YCIV3Lklhh72aCFlCowRKLmQJorUSJa5AN41gbD64SzTIMHeIlUe/LzxiLF3Qsml0neKHH37Irl27ePHFF4mJiWHatGn06tWL6667jsWLF+N0OptznILjJKnGjWkxapE1klpWAYEJe/fqfK7uGscFyYF6iqCU1V/XFWCN1PLAOXEYZCls3VRXU0D149Ye9RVvJqwpYFellg4GHwPjAoZPrJQFLZW8apl9FQpXLcthda6TORuLGZRspJtJzxkmPe10Ev/8nwNzpBaDDPd9G+o5mZllY+El7YnRS2rv0SDBWLqgddBoowgBQe3bb7+dL7/8kq1btzJlyhTy8vIYP3483bp1Y8KECaxdu7a5xio4DmQpkE33xs/FePzhM+YKq7zMGGzlufMtZPY1U1rtZU+pl7tXFzAgyci+skBrqboF/tMyLBwocwMgSeHf92Bl6I+DQNBSyXdC5rpAQtmAxAjuPSee6z7PZeyXB7n96zz8SNzf18zsbBsd24VXeCqu8rGzxC0K71s5x2QU69KhQwf++te/snjxYq6++moqKir417/+xTXXXEPv3r2ZP38+Pp9YHZ1q8ip9zNtWQg9zpLprrEsgY87D5Z8dwIfEx785KHXXroALqnxoJFi4PVCnODU9gcy+ZhZuL6HCEzhPlsIrf7SParR3XiA4pZS5/ZgNMhPOied/h9zctyZ0Jzgjy0ZxlY+x3UxUef1h7/d2ERoW7XLUW0C+PSJZJI+1Io7rV6u8vJzPPvuMjz/+mB9++AFZlhk9ejQ333wzer2ed999l6lTp/Lrr7/y6quvNvWYBceANVJWs+lSohN5c3gyd6/Or5cxF3QBfTAqhYfWFgKBCf1rSTXnJRv5S+/QLhnPDE6kUzst0zISuOqMKM6M1XHv6oKQmGL3KLFTFLQO2kcFSo5mZtl45SJrgzH0TlFaTHpJVYSqW7uYbIBZgyxMXR8QxpA1cH6yMUQTWNDyabRR9Pl8rFy5ko8//pjly5fjdDo599xzmT17Ntdffz3x8fHquZdccglPP/00b775pjCKp5hg2cPU9Ta6xEYQHyHxyehUCqq87C31hLR/cnoVdpW4ya0IFTruaY7grDgd712SQllNpt3fN9u5raeJWdnFDEmO4Kr2Gs66tgMHK720j9KGrV0UCFoq3aK8nBWvx+lViNFpwpYT9YzXs9fhRqMBv+Ljw1EpbLZVowF6mrSkRPhISdbQY3SKyKZuxTTaKHbr1o1Dhw5htVoZP348N998M927d2/w/B49elBRUdEkgxQcP8Gyh86XtWfNQSfT19voFa9n1uAkVdMxiFEr0dmkZ2p6Qoik1S/F1czdYg/poWjUSihKbbxEwsdZUV7OigIQO0RB60IC+sRpMWol9pe5eXmolQfXhtYgPvp9Ibf3imPC6gI+HJVCjE5Df0sEKYctAkU2deum0UZxxIgR3HzzzVx00UWNKuK/7rrruO66605ocIKmQQLK3YHaQqdXYWNRNX9ZlVdP3u3loVZA4cXN9gbdq0Gt02AzYlFsL2grpNXxqswcnBi2z6jTq+D0KpS5FW5ZkStqEdsgjTaKb731VnOOQ9DMVHj8YeXdPh6dSoXHT16FhxlZNgwyvHNxCj5F4Y+y+u7VvpYI1lzbAZfXyxUdRPNgQdtBAnrH6xjfO46kSC13bMmv50kJekf2lrpFLWIb5bgTbcrKyvD7/fWeS0tLO+FBCZqejtGhcZLUaC3jeppw+xQMsoTXHyhQfm+ngwitxBnRWu5cVf9HoXusTAdD0D0qfgAEbQub04cXifGr8uq1Swt6R+YOsfJ0ti3kdU6vwv5KvxCybwMck1F89913efXVV/njjz8aPKekpKTB5wSnjjSDjzeHW7m7Rtx74jmh2aTTMiws2nWIB/vGIykK/3N4WHZlKveszlc7jLc2V+nR+jkKBIdj0GmZmRVQqgm2SwtkkUZS4fGS2S9Qx2t3hc4Do1YiWqtBwSfuuVZOo43i+++/T2ZmJsOGDeNPf/oTM2fOZMKECRgMBj766COSkpIYP358c45VcAJIQLpFx8ejU6n0+nn8h0I1ZgKBOsSx3Uz1NE3fHpFMapREfETryqQTDYwFx4oCHCj3qt6RYLs0gKnpCby42c4Ho1J4Z0dx2HZryUZxz7UFGl28/8Ybb3DRRRexZMkSbr/9diBQejFt2jR++uknHA4HZWVlzTVOQRNQ6pEY+2Uu+0vd3NE7nrlb7MzKLubFzXbu6B1PrD7gXg06xZ1ehbu+ySc+Qm51Mm0BIfSCenGfHJdQFhGEJ8cls7csvCKNogTuoW3F1cwYnMhZcTpmD7bw7iUpTM9IoKdJi9uPuOfaAI02inv37mX06NGBF2kCL/N4PACYTCZuvfVWFixY0AxDFDQVwey5Du309TRQZ2bZ6NBOr/4ABHF6FX4r86M08J4tlSP1cxQIwlFY5cNQs+s7XNLw/V8dGLUSPj/8kOfk/5bnUeWDp7OKGNY+kiGJkrjn2giNdp9GRUWh1PxaRkdHI8sy+fn56vPx8fHk5eU1/QgFTUZqdKAOa7fDHXby7nW4mZ5hYV6dNlJGrcQ2m4sKj470BB3JEa1jxygaGAuOFWukzP5KP3M22ZiankD7KB17S91qvW4w0Wbm4CQe7mdWQw7lbi9SlLjn2gqN3il269aNXbt2AaDVaunTpw+LFi3C4/HgcrlYtGgRHTt2bLaBCk4MBSio9DBjkAWvXwnrIrowNZJO7bRqEkFwlfzOTgf3ri7gX3uq+Dq/dewag0o+dVf8rS1RSHBySTX4SIsO3P+P/2hj2voifAqM62Vi4cUpLNxewl9qeisGQw6miFqxb3HPtQ0abRRHjx7N8uXLcblcAEyaNIkff/yRTp060aVLF7KyssjMzGy2gQpOjByXzE0r8lm0q5T0pAheGlp/8pp1CtN+LGLhxSmq8HfdomU/hI2RKMABl8yGksDntASjWbeB8cqr25N1Q5pIeBAcEQlIMGhU92luhZe5W+y0j9LRTi9xc3cT8+rMh5lZNgYkGlWjJ+65tkGj3af3338/999/v/r48ssv54svvmDp0qXIssyll17KBRdc0CyDFJw4wXjH9V3a8YvdzVlxOj69Io28Sg+ROg2RsoLN6WNPqZft9mpV1SY1WstjAxOQNdArPgKzQabQ6VPrsVpylqdoYCw4VsqrvUTqJCb1M+MnsGswaiXsLj8zs4tDznV6FXz+0HCCuOdaP0c1ii6Xiy+//JL9+/djNpu55JJLsFqtAAwaNIhBgwY1+yAFJ441UqZLrBaTQcucTTbu6B3PzKz8kJTyS9KMGLUS7+10qPGTwHmhqecpkbUxkkCWZ55Q9xC0CcwGmW8PHOLecxI4UO4hSqdh3tZiJg9MFPHC04QjGsX8/HxGjx7N/v371SSbqKgo/vWvf3HhhReelAEKmoZUg4/XhiVzzee5ZPY118s+nZFlo68llX+PSuamFfnM31bCS0OT+POKvHrnXd6hVrXoSBl3Qt1D0NpINfi4s4+Z7/Kq8CuwNqech/olsKvExeLLU3nw29YrZiFoHEc0ik8//TQHDhxgwoQJDBkyhL179/LCCy/wt7/9jR9//PFkjZFDhw7xzDPP8O2335KTk4PZbGbUqFE8/vjjIS2rHA4HkydPZvny5QBceumlPP/885hMJvWcHTt28Mgjj7B582bi4uK4/fbbmTx5cqNEzlszEiArAf3TBKMmpHD/vZ0Ociu85FZ4SbfoyLohjUKnD5dPOqrBExl3graCAnyRp3DXNwGh7y6xWiYPsKgLw9YsZiFoPEc0it9++y0333wzTz/9tHosMTGRO++8k4MHD9K+fftmHyAEdqz5+fk89dRTnHXWWeTl5TFp0iT+8pe/8Omnn6rn3XnnneTm5rJ48WIAHnjgAe6++24WLVoEQFlZGddccw2DBw9m9erV7N69m4kTJxIZGRkSL22LKICk0XBBcgRxBpmpP9bXdCyo8lLo1DAwDjoYIMcVvq+cDw0KgcapwYy7YNGyWEELWis5Lpm7vslR7/ex3Uxq+yioFbMQoYG2zRGNYmFhIRkZGSHHzjvvPBRFITc396QZxZ49e/Lhhx+qj88880xmzJjBjTfeSFlZGe3atWPXrl2sWrWK5cuXk56eDsDcuXO57LLL2L17N127duWTTz7B6XTy+uuvYzQa6dmzJ7/99hvz58/nvvvua9O7xRyXzOYiJ9PPS+KqZTkhE31mlo1Fo1N5eG0+AxKTQwze2yOSueub/BADet+afJaMTlFVboIZd6KxqqA1c3gowKA9uqfkSPq6Qnu3dXLEkgyfz4fBEBoYCj4OlmacKsrLy4mIiCAyMhKA7OxsoqOjQ4z4eeedR1RUFFlZWeo5gwYNwmg0queMGDGC/Px89u/ff3Iv4CRTWOWj1KOQU+EJO9FLXF7u6B3PfWvy1ZILCUiNlMjsaw4p0dhT6g1R6Qhm3AV2mGLiC1onwVBAkG4mfdh63hh9YC8RzLzO+CSHiz87SPonOWod75GeE7Rsjpp9um/fPjZt2qQ+Duqb7t69m+jo6Hrn9+/fvwmHFx6Hw8GsWbO49dZb0WoDl1BUVITZbA7Z7UmSREJCAkVFReo5KSkpIe9lsVjU5zp16tTsYz9VWCNlZAmidQGXqNkgc2sPE5IEsgRp0Toe/9FGboU3ZCVsNsjM3WIXMUNBm+fwUMCBcnfY9lEub6B12pEyrwGRld1KOapRnD17NrNnz653fPLkySGPFUVBkqRjah319NNPM2fOnCOes2zZspBM14qKCm6++WaSk5OZMWNGoz/rRNi9e/cxHW+JaHU6+phSKHX7eGtEMrkV3hCVf1NEYHdo1Ep4/PDHvn14PR60Oh1vDU1m/Fq7eu5bQ834Cn5nd432bUuhNX0fR6OtXEtru45uOh0rruzAyjwPHWL0PP5jbTcZRQl0kxlqimb37jzyjO3Del3+sJdDjYB4uOeqqw6ezEsKobV9H0fiRK6la9euDT53RKM4b9684/7QxnDvvfcyduzYI56Tmpqq/ruiooIbbrgBgEWLFoW4dhMTE7Hb7apxhoChLi4uJjExUT3HZgttDhp8HDwnHOH+gME4ZWuiE5BXLXPIIzH+m/yQVezUH4uY1M+MUafhgbWFLBndmTNqVrSdIEzMsNMpuorwtMbvoyHayrW01utQgHynjqk/Ftar0313pJWeiRqkxK4YXDJGraOeF+UMcwwSNPhch/an5m/SWr+PcDTntRzRKP7f//1fs3xoELPZjNlsbtS55eXl3HDDDSiKwuLFi+u5btPT06moqCA7O1uNK2ZnZ1NZWak+Tk9P58knn8TlcqkGdc2aNSQnJ58Wuq0S0D7CR15l+FVsx3Y6nvypvgtVqHQITieCyWM9RqdQUu1jxZVWqhVNvSSyo2Vezx9mZcKa2ufmDxNZ2a2BRsu8nUrKy8u59tprKS8v56OPPqKqqoqqqioA4uLi0Ov1dO/enZEjR5KZmclLL70EQGZmJqNGjVJXFNdffz3PPfccEyZMYNKkSezZs4eXXnrptKhTrEtSA7WFFqOGe86Oo9qnEKPXouAVSTOC05K6C8Hdu3/n7K5dOXxBeHjmdUqkjE+BjSU+YiK0/GN7QYjrdVa2jf4JKSKm2MJpFUZx69atbNiwAaifyFM35rhgwQImT57MddddB8Bll13G888/r54bGxvLp59+yqRJkxg2bBgmk4mJEydy3333naQraRnIEmE7h2skmJVdjNOrMGeT/Zg0TEX6uaAtEO4+PhJB45lmgK/zPSG7xmkZFlVQP4hQemr5tAqjeOGFF+JwOI56nslk4q233jriOb169eKrr75qopG1TvIqfczbVhKyip23rYRxvUzHlS3XkkXBBYLG0tB93E2nO+prw2WizsyykdnXzDMbAkLiImu7ddAqjKKgabFGythdPnWyAmpX8bo0VsNUiIIL2gIN3cerRgXKtoK7yP0VfqL1GpIiUJtuN6QBLNdUggulp9aDMIqnIeESBOYOsTJnU2hmbmNXtkIUXNAWaOg+LvZIYXeR0zMs9DRpGZIoNagBfHnHKIYkRwilp1ZEo5sMC9oOwQSBjWPT+OqqNF69yEqV18cDfc1hu4YfrYHw4UogwdcHO5ILBK2Bhu7jBJ1Ss4sM1UGdkWVjo81NjktWF5qHz5/uUV6h9NTKEDvF05jtJaGJAQtGJLNxbBp7K/xEazUkG4PJA0eOFQpRcEFboKH7OLLqAIXaqLC7SD+1yTNCA7htIIziaUqOS2bq+sKQFlJP/FTEa8OS2VDoYuEOB+N6mpiz2d6oWGHveB1fjkmjwuunY5RG/CAIWh0Nidvv2e1p0D2qATXEIOp52wbCKJ6mlLh89dQ65g6xssdRzQub7EzLsCCjNKpLQLiMvbRk4ZkXtD4aMmzhdpHBmKLwiLQtxC/XaYpBp1UNIgSMXea6AjrERGA2yMzMstGhXfguAXVjheFiLbevKlA7bQgEbYG6u8hlY1L5ckwaYzoaGJIoCY9IG0MYxdMEBUISZsqrvWF3gRuKnNzaI1CvuNvhZnqGJWzyTZAjZZ4KBG2J4C7yQrNC31gf7SNEiKAtItynpwHhXJz/vSI1bIzE5wdJqumW4VM4J153xOSBhmItokhZIBC0RsRO8TQgnItz4pp83hqRHLILnJZh4ePfHGiAN4ZbualLJEMSpSM2EG4oFV3EWQQCQWtE7BRPA8K5OPeUejkzRuI/o1NYX1iNzx/oFTctw8KABB0pET6kRuz0GsrYE24lgUDQGhFG8TSgIRdnjE4m0v47N5zZmUKnj5u6pNQYtMbt8kLFk2FAHI1+rUAgELREhPv0NOBILk6vx3NE92hDBOOUGZ/kcPFnB0n/JIev8/1hFW8EAoGgtSB2iqcB4VycshTo+xYZm4JSc86xIETABQJBW0QYxdOEI/V9e3ek8ZjbPAkRcIFA0BYR7tPTjKYqthci4AKBoC0ijOJpxtGK7Q8v8m8oRihKMQQCQVtEuE9PM45UbK/gC6tjGs61KkoxBAJBW0TsFE8zjrTDO1bXajBOKfrFCQSCtoLYKZ5mHL7Di/RU0jMxsBMUyTMCgeB0R+wUT0Pq7vD0pXnqDk8kzwgEgtMdYRQFKiJ5RiAQnO4I96lARSTPCASC0x1hFAUhNNR5XCAQCE4HhPtUIBAIBIIahFEUCAQCgaAGYRQFAoFAIKhBGEWBQCAQCGqQHA6HaIEnEAgEAgFipygQCAQCgYowigKBQCAQ1CCMokAgEAgENQijKBAIBAJBDcIoCgQCgUBQgzCKYaiuruaRRx7hzDPPJCUlhZtuuomDBw8e8TV9+vTBZDLV+2/s2LHqObNnz673fLdu3VrUdTRmjIqiMHv2bM466yysViuXX345v/76a4u6jhdffJFhw4aRlpZG586dufHGG9m5c2fIOffee2+9ax05cmSTjn3BggWcffbZJCUlMXToUH788ccjnv/9998zdOhQkpKSOOecc1i4cOEJv2dTcCyfuXTpUq655ho6d+5MamoqI0aM4Msvvww556OPPgo7X1wuV3NfyjFdy3fffRd2nL/99lvIeZ999hkZGRkkJiaSkZHBsmXLmvsyjuk6wt3rJpOJlJQU9ZzGXmtT8sMPP3DTTTfRo0cPTCYTH3300VFfs2PHDkaPHo3VaqVHjx4899xzKEpoEcWJfB/CKIZhypQpLFu2jH/84x98+eWXlJeXc+ONN+LzNawFumbNGnbt2qX+t3btWiRJ4uqrrw45r2vXriHnNecP2vFcR2PG+PLLLzNv3jyee+45Vq9ejcVi4ZprrqG8vLzFXMf333/PX/7yF1asWMHSpUvRarVcffXVHDp0KOS8iy66KORaP/nkkyYb95IlS3j00Uf561//yrp160hPT+eGG24gJycn7Pn79u1j7NixpKens27dOh5++GEmT57MZ599dtzveSqu44cffmDIkCF8/PHHrFu3josvvphbbrml3n0UGRkZ8rfftWsXBkPzNu483r/fTz/9FDLOzp07q89lZ2dzxx13cMMNN/Ddd99xww03cPvtt7Nx48YWcx3PPvtsvb91p06d6v0+wZGvtamprKykZ8+ePPvssxiNxqOeX1ZWxjXXXENiYiKrV6/m2Wef5dVXX+W1115TzznR70PUKR5GaWkpXbp0Yd68eeouLzc3lz59+rB48WJGjBjRqPeZM2cOr7zyCrt27VK/7NmzZ7N06VLWr1/fbOMPcrzXcbQxKorCWWedxV133cWkSZMAcDqddO3alZkzZzJu3LgWcR2HU1FRQYcOHfjoo4+47LLLgMDquaSkhEWLFjXpmIOMGDGCXr168corr6jH+vXrx1VXXcUTTzxR7/wnnniCZcuWsXnzZvXY/fffz//+9z9Wrlx5XO95Kq4jHMOHD2fQoEHMmjULCOwUJ0+efNQdf1NzrNfy3XffceWVV/L7779jNpvDvue4ceM4dOgQ//3vf9VjV111FQkJCfzjH/9o8muAE/9OfvrpJy699FJWrFhBRkYG0LhrbU7at2/P888/z5/+9KcGz/nHP/7Bk08+yW+//ab+rr7wwgssXLiQnTt3IknSCX8fYqd4GFu3bsXj8TB8+HD1WGpqKt27dycrK6tR76EoCh988AE33nhjvdXPvn37OOusszj77LO544472LdvX1MOX+VEruNIY9y/fz+FhYUh72s0Ghk8eHCj/z4n6zrqUlFRgd/vx2QyhRxfv349Xbp0oX///jzwwAPYbLYmGbfb7Wbr1q0h44aAcWho3NnZ2fXOHzFiBFu2bMHj8RzXe54oTfWZFRUV9f72TqeT3r1707NnT2688Ua2bdvWFENukBO5losuuoju3bszZswY1q1bF/Lchg0bwn5vLfk7ee+99+jRo4dqEOtypGs91WRnZzNo0KCQ39URI0aQn5/P/v37gRP/PoRRPIyioiJkWa63UrJYLBQVFTXqPdasWcP+/fu59dZbQ44PGDCA+fPns3jxYl555RUKCwu55JJLKCkpabLxBzne6zjaGAsLC9X3OZb3PV6a4vsAePTRR+nTpw/p6enqsZEjR/LGG2/w2Wef8fTTT7Np0ybGjBlDdXX1CY/bbrfj8/mO6e9UVFQU9nyv14vdbj+u9zxRmuIz3377bfLy8rjxxhvVY127duW1117jn//8JwsWLCAiIoJLL72U33//vUnHX5fjuRar1cqLL77IBx98wAcffEDXrl256qqrQlzBhYWFreo7KS0t5b///W+936fGXOuppqE5EnwOTvz7OG36KT799NPMmTPniOc0VXD8vffeo1+/fvTp0yfk+MUXXxzyeMCAAZx77rn885//5L777mvUezf3dTTFGBvDyfw+HnvsMX766SeWL1+OLMvq8euuu079d69evTj33HPp06cPK1asYMyYMU3y2ac7n332GdOnT2fhwoV06NBBPZ6enh6yQMnIyODCCy/kzTff5Pnnnz8VQw1L165d6dq1q/o4PT2dAwcO8MorrzB48OBTOLLj5+OPP8bv93PTTTeFHG+L13o8nDZG8d577w3JBA1HamoqGzZswOfzYbfbSUhIUJ+z2WwMGjToqJ9js9n48ssvj/qDDxAdHc1ZZ53F3r17j34BNZys62hojElJSer7pKWlhbxvYmJii7uOKVOmsGTJEpYtW0anTp2OeG5ycjIpKSnH9H00hNlsRpbleu7YI/2dEhMTw56v1Woxm80oinLM73miHM91BPnss8+45557eOONN9Q4bkPIssy5557bJH/7hjiRa6lL//79WbJkifo4KSmp1XwnEFi0jxkzhri4uKOee/i1nmoamiPB5+DEv4/Txn1qNpvp1q3bEf+LjIzk3HPPRafTsWbNGvW1Bw8eZNeuXWH974fzz3/+k4iIiJBdSEO4XC52796tGpqWdB0NjbFjx44kJSWFvK/L5WL9+vXH9L4n4zr+9re/8Z///IelS5c2qvTFbreTn59/TN9HQ+j1es4999yQcUPAtd7QuNPT08Oe37dvX3Q63XG954lyvJ/56aefcvfddzN//nyuuuqqo36Ooijs2LGjSf72DdFUf79ffvklZJwDBw5sFd8JwKZNm9i+fXs912lDHH6tp5r09HTWr18fUrqzZs0akpOT6dixI3Di38dps1NsLLGxsfz5z3/miSeewGKxEBcXx9SpU+nVqxcXXXSRet7AgQO56667GD9+vHpMURTef/99rr32WqKjo+u99+OPP86ll15KamoqxcXFvPDCC1RVVXHzzTe3mOs42hglSeLee+/lxRdfpGvXrnTp0oU5c+YQFRXF9ddf32KuY9KkSSxatIgPP/wQk8mkxkKjoqKIjo6moqKCZ599ljFjxpCUlMSBAweYMWMGFouFK664oknGPnHiRO6++2769+9PRkYGCxcupKCgQM3QvfvuuwF48803gUAW49tvv82jjz7KuHHjyMrKUmNujX3P5uBYr+M///kPd999NzNnzmTw4MHq316v16u7k2effZaBAwfSuXNnysrKePPNN9mxYwcvvvhis13H8VzL/Pnz6dChAz169MDtdvPxxx/zxRdf8P7776vvec899zB69Gjmzp3L5Zdfzueff853333H8uXLW8x1BHn33Xfp3LkzF154Yb33bMy1NjUVFRWqd8Dv95Obm8vPP/9MXFwcaWlpPPXUU2zatImlS5cCcP311/Pcc88xYcIEJk2axJ49e3jppZeYPHkykiQBJ/59CKMYhtmzZyPLMuPGjcPlcjFkyBDeeOONkHjU7t27sdvtIa/77rvv+P3333nrrbfCvm9eXh533nmn6gocMGAAK1euDIm1nOrraMwYH3zwQZxOJ4888ggOh0N1scTExLSY6wgaksN3KX/729+YMmUKsiyzc+dO/v3vf1NaWkpSUhIXXngh77zzTpNdx7XXXktJSQkvvPAChYWF9OjRg48//lj9W+bm5oac36lTJz7++GMee+wxFi5ciNVq5bnnngu5hqO9Z3NwrNexcOFCvF4vU6ZMYcqUKerx888/ny+++AIIJHs8+OCDFBUV0a5dO84++2y+/PJL+vfv32zXcTzX4vF4mD59Onl5eRgMBvX8Sy65RD0naJSefvppnnnmGc444wwWLlzIgAEDWsx1AJSXl7Nkyf+3d/8xVdV/HMefMDC8NrwpoiXJ9NrVKSKzSTjC2UBNHVO5ord0KnQZuiZFKcPF5A43buHCKUIDxdXSiVSCU2ByRSPBrEn5A6fTREoj/iCFe8uGdb3fP7j3fO8NriCoWb4fG3+cD+d+zucctvvinM/nfD4HSEtL67HOvpzrg/b9998TGxurbJtMJkwmE6+99hofffQRra2tXLt2Tfn90KFDKSsrY/369bzyyiuo1WrefPNNt/EOA/17yHuKQgghhMMT06cohBBC9EZCUQghhHCQUBRCCCEcJBSFEEIIBwlFIYQQwkFCUQghhHCQUBTiMeVciNc5+z90TY/39zl1Hyf30z61Wo3JZHps2iMESCgKAXheCd75c/To0X+6iffFZDK5tT8gIIApU6aQlpZGe3v7P908IR5bMqONEC7S09MZO3Zst/KQkJBH3ha9Xo9Op+Opp57qdx1btmzB39+f27dvU1tbS1FREefOnXtoU5Bt376du3fvPpS6hXgUJBSFcBEdHc306dP/6WYAXStHuE5l1x/OuV2ha27VxMREDhw4QENDw0OZUs3X1/eB1ynEoySPT4W4T2q1mtTUVA4ePEhERASjRo0iOjqac+fOAf9fT3PkyJHMnz+f5uZmt8+fPHmShIQEQkJCCAwMZMKECaSkpHDr1i23/XrqUxwo53JbrvNJAnz33XfEx8czZswYRo0axauvvtpt1fXffvuNjIwMQkNDCQwMRKPRsGDBAurr65V9eurD6+zsZOPGjWg0GoKCgtDr9fz888/d2uap/8/5KNjV3r17WbhwIVqtlsDAQKZNm0Zubm6f7lJra2uZN28ewcHBPPvss4SFhbFhw4ZePyeeDHKnKIQLi8XSbaJ36FrqytU333xDdXU1BoMBLy8vcnNz0ev1rF+/nsLCQhITE+no6GDbtm2sXbuWqqoq5bPl5eVYLBZWr17NiBEjaGxs5NNPP+XixYtUV1crs/0/DD/99BOAW8jU1dWh0+mYMmUKGzZswNfXl/379xMXF0dZWZmyosI777xDeXk5BoOBiRMn0tHRwenTp2lsbCQyMtLjMdetW0dpaSnx8fGEh4dTV1fX61qavdm1axdarZbZs2fj5+dHbW0tWVlZWCwWjEajx89dunSJpUuXMmnSJNLT01GpVFy7do2ampoBtUf8d0goCuHC0zqYra2t+Pn5KdtXrlzh22+/Vfof1Wo1b7/9NtnZ2TQ0NDB06FAAbDYbubm5NDU1MW7cOACMRiMqlcqt/vDwcJKSkjh16tR9LQLdm1u3buHj48Pvv//OV199xa5duwgICFBWUrfb7aSmphIREUF5ebkSyImJicycOZPNmzdTXV0NwJEjR1i1ahXZ2dl9Pv758+cpLS3ljTfe4MMPPwQgKSmJ5ORkLly40O/zqqiocLuGBoOBt956i507d7Jx40aP/bDHjx+ns7OTzz//3O0fnXsFqXiySCgK4eKDDz5gwoQJ3coHDRrkth0VFeU2IMfZPxcbG6sEomt5c3OzEorOL3O73Y7VauXPP/8kPDwcgDNnzjzQUIyIiHDbnjx5Mvn5+Uobzp8/z5UrV0hJSeHmzZtu+86aNYuioiJu376NSqXC39+f06dP09LSwnPPPden45vNZqArCF2tWbOG/fv39/e0lPbbbDasVis2m43IyEg++eQTLl++7PE1DH9/f6ArVFesWIG3t/QgCXcSikK4mDZtWp8G2gQFBbltO79sR48e3WO562sQN27cYNOmTZjNZqxWq9v+FoulP8326OOPP0atVvPrr79SVFTEjz/+6HaHdfXqVaDrEee6det6rOPmzZuoVCo2b97M2rVrCQkJITQ0lJiYGJYtW8YLL7zg8fjXr1/Hy8tL+YfASaPRDOi8vv76a7KysmhoaODOnTtuv7vXNYyLi2PPnj2kpKRgNBqZOXMmCxYsYPHixfj4yNehkFAUol88jQr1VG63dy1barPZiIuLo62tjdTUVLRaLUOGDOHu3bvodLoH/jrDjBkzlNGn8+bNIzIyEoPBwJdffom3t7dyPKPRSFhYWI91BAQEALBo0SJmzJhBZWUlx44do7CwkG3btlFQUEB8fPyA2+qpL9Vms7ltNzc3s2jRIjQaDdnZ2QQFBeHn58fZs2fJzMy85zUcPHgwFRUV1NfXYzabqampISkpifz8fKqqqhg8ePCAz0P8u0koCvEIXbhwgcuXL1NQUMDrr7+ulDvv2B4mlUpFeno6ycnJlJWVodPplEfATz/9NLNmzeq1jpEjR5KQkEBCQgLt7e3Mnj0bk8nkMRSff/557HY7TU1NTJw4USnv6XzVajUdHR3dyq9fv+62XVlZSWdnJyUlJcpK80CfR+l6e3sTFRVFVFQUWVlZFBcX8+6773Lo0KEBDwAS/37yQF2IR8h5J+m8c3TKy8t7JMfX6XQEBQWxdetWAMLCwhg3bhz5+fndHuUCtLW1AV13a38PLLVaTXBwcI9B5hQTEwPAzp073coLCwu77Tt27FgsFguNjY1KWWtrKxUVFW779XQNOzs7KSoq8tgOp7/3mwJMnToV4J7nIZ4ccqcohIuamhqampq6lb/44ouMHz9+wPVrtVo0Gg0ZGRm0tLTwzDPPYDabaWlpGXDdfeHj48OaNWvIyMigurqaOXPmkJeXx5IlS4iIiGD58uWMHj2aX375hfr6eux2O4cPH8ZqtTJp0iRiY2MJCQnB39+fU6dOcfTo0W6DaFyFhoayZMkSiouLsVgsvPTSS5w4cYIffvih2746nQ6j0ciKFStITk7mjz/+YPfu3Wg0Gs6ePavsFx0dzaBBg9Dr9axevZo7d+5QUlLSp0EzOTk51NXVMXfuXMaMGUN7ezu7d+9myJAhzJ07t38XVfynSCgK4eL999/vsTwnJ+eBhKKvry8lJSWkp6eTl5eHt7c3MTExfPHFF2i12gHX3xerVq0iJyeHrVu3MmfOHCIjIzGbzWzZsoXi4mKsVqvyQvzKlSuBrkevBoOB48ePU1VVxV9//UVwcLAy+OZeduzYwfDhw/nss8+orKwkKiqK0tJSJk+e7LbfsGHD2LNnD++99x6ZmZkEBwezadMmrl696haK48ePZ+/evWRlZZGZmcnw4cPR6/W8/PLLLF68+J5tmT9/Pjdu3GDfvn20tbUxbNgwpk+fTlpamtujWPHk8mpvb7f3vpsQQgjx3yd9ikIIIYSDhKIQQgjhIKEohBBCOEgoCiGEEA4SikIIIYSDhKIQQgjhIKEohBBCOEgoCiGEEA4SikIIIYSDhKIQQgjh8D98AkqJcTMLzQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.scatterplot(\"res_email\", \"res_payments\", data=residuals)\n",
    "plt.plot(np.linspace(-0.7, 1), model_treatment.params[0] + np.linspace(-1, 2) * model_treatment.params[1], c=\"C1\")\n",
    "plt.xlabel(\"Email Residuals\")\n",
    "plt.ylabel(\"Payments Residuals\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "말씀드렸듯이, 교육 목적을 위해 이렇게 계산했습니다. 실제로 회귀 모형에 treatment와 control 변수를 추가하면 추정치는 정확히 동일합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "        <td></td>          <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th>    <td>  490.8653</td> <td>    9.715</td> <td>   50.527</td> <td> 0.000</td> <td>  471.820</td> <td>  509.911</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>email</th>        <td>    4.4304</td> <td>    2.130</td> <td>    2.080</td> <td> 0.038</td> <td>    0.255</td> <td>    8.606</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>credit_limit</th> <td>    0.1511</td> <td>    0.008</td> <td>   18.833</td> <td> 0.000</td> <td>    0.135</td> <td>    0.167</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>risk_score</th>   <td>   -8.0516</td> <td>   38.424</td> <td>   -0.210</td> <td> 0.834</td> <td>  -83.379</td> <td>   67.276</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_2 = smf.ols('payments ~ email + credit_limit + risk_score', data=data).fit()\n",
    "model_2.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "결론적으로, outcom을 예측하기 위한 좋은 통제변수가 있을 경우 굳이 교란요인이 아니더라도 모델에 추가하는 것은 좋은 생각일 수 있습니다. treatment 효과 추정치의 분산을 낮추는 데 도움이 됩니다. 이 상황을 인과 그래프에서 어떻게 나타낼 수 있는지 보여줍니다.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n -->\n<!-- Pages: 1 -->\n<svg width=\"448pt\" height=\"116pt\"\n viewBox=\"0.00 0.00 447.80 116.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 112)\">\n<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-112 443.8,-112 443.8,4 -4,4\"/>\n<!-- X -->\n<g id=\"node1\" class=\"node\">\n<title>X</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"27\" cy=\"-90\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"27\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">X</text>\n</g>\n<!-- Y -->\n<g id=\"node2\" class=\"node\">\n<title>Y</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"63\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"63\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">Y</text>\n</g>\n<!-- X&#45;&gt;Y -->\n<g id=\"edge1\" class=\"edge\">\n<title>X&#45;&gt;Y</title>\n<path fill=\"none\" stroke=\"black\" d=\"M35.35,-72.76C39.71,-64.28 45.15,-53.71 50.04,-44.2\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"53.23,-45.64 54.7,-35.15 47.01,-42.44 53.23,-45.64\"/>\n</g>\n<!-- T -->\n<g id=\"node3\" class=\"node\">\n<title>T</title>\n<ellipse fill=\"none\" stroke=\"gold\" cx=\"99\" cy=\"-90\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"99\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">T</text>\n</g>\n<!-- T&#45;&gt;Y -->\n<g id=\"edge2\" class=\"edge\">\n<title>T&#45;&gt;Y</title>\n<path fill=\"none\" stroke=\"black\" d=\"M90.65,-72.76C86.29,-64.28 80.85,-53.71 75.96,-44.2\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"78.99,-42.44 71.3,-35.15 72.77,-45.64 78.99,-42.44\"/>\n</g>\n<!-- email -->\n<g id=\"node4\" class=\"node\">\n<title>email</title>\n<ellipse fill=\"none\" stroke=\"gold\" cx=\"175\" cy=\"-90\" rx=\"30.59\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"175\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">email</text>\n</g>\n<!-- payments -->\n<g id=\"node6\" class=\"node\">\n<title>payments</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"276\" cy=\"-18\" rx=\"44.69\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"276\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">payments</text>\n</g>\n<!-- email&#45;&gt;payments -->\n<g id=\"edge5\" class=\"edge\">\n<title>email&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M194,-75.83C208.6,-65.72 229.04,-51.55 245.87,-39.89\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"248.06,-42.62 254.28,-34.05 244.07,-36.87 248.06,-42.62\"/>\n</g>\n<!-- credit_limit -->\n<g id=\"node5\" class=\"node\">\n<title>credit_limit</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"276\" cy=\"-90\" rx=\"51.99\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"276\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">credit_limit</text>\n</g>\n<!-- credit_limit&#45;&gt;payments -->\n<g id=\"edge3\" class=\"edge\">\n<title>credit_limit&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M276,-71.7C276,-63.98 276,-54.71 276,-46.11\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"279.5,-46.1 276,-36.1 272.5,-46.1 279.5,-46.1\"/>\n</g>\n<!-- risk_score -->\n<g id=\"node7\" class=\"node\">\n<title>risk_score</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"393\" cy=\"-90\" rx=\"46.59\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"393\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">risk_score</text>\n</g>\n<!-- risk_score&#45;&gt;payments -->\n<g id=\"edge4\" class=\"edge\">\n<title>risk_score&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M368.76,-74.5C351.42,-64.12 327.89,-50.05 308.87,-38.66\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"310.59,-35.62 300.21,-33.49 307,-41.62 310.59,-35.62\"/>\n</g>\n</g>\n</svg>\n",
      "text/plain": [
       "<graphviz.dot.Digraph at 0x12cab7470>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g = gr.Digraph()\n",
    "g.edge(\"X\", \"Y\"), g.edge(\"T\", \"Y\")\n",
    "g.node(\"T\", color=\"gold\")\n",
    "\n",
    "g.node(\"email\", color=\"gold\")\n",
    "g.edge(\"credit_limit\", \"payments\")\n",
    "g.edge(\"risk_score\", \"payments\")\n",
    "g.edge(\"email\", \"payments\")\n",
    "\n",
    "g"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mostly Harmful Controls\n",
    "\n",
    "두 번째 동기 부여 예시로 두 병원의 약물 검사 시나리오를 생각해 봅시다. 둘 다 특정 질병을 치료하기 위해 신약에 대한 무작위 실험을 진행하고 있으며, 반응변수는 입원 일수입니다. 치료가 효과적이면 환자의 입원 일수가 줄어들 것입니다. 한 병원의 경우, 치료와 관련된 정책으로 임의로 환자의 90%에게 약을 투여하고 10%는 위약을 투여합니다. 다른 병원의 경우, 다른 정책을 가지고 있습니다 : 환자의 10%에게 무작위로 약을 투여하고 90%는 위약을 투여합니다. 환자에게 90%의 진짜 약물과 10%의 위약을 투여하는 병원이 보통 치료하는데 더 심각한 질병인 경우가 많다고 합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>hospital</th>\n",
       "      <th>treatment</th>\n",
       "      <th>severity</th>\n",
       "      <th>days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>29.686618</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>20.050340</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>20.302399</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.603118</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.332793</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   hospital  treatment   severity  days\n",
       "0         1          1  29.686618    82\n",
       "1         1          1  20.050340    57\n",
       "2         1          1  20.302399    49\n",
       "3         0          0  10.603118    44\n",
       "4         0          0   8.332793    15"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hospital = pd.read_csv(\"./data/hospital_treatment.csv\")\n",
    "hospital.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "랜덤화된 데이터를 다루고 있기 때문에, 첫 번째 직관은 단순히 treatment를 outcome에 대해 회귀하는 것입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>   33.2667</td> <td>    2.662</td> <td>   12.498</td> <td> 0.000</td> <td>   27.968</td> <td>   38.566</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>treatment</th> <td>   14.1533</td> <td>    3.367</td> <td>    4.204</td> <td> 0.000</td> <td>    7.451</td> <td>   20.856</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hosp_1 = smf.ols('days ~ treatment', data=hospital).fit()\n",
    "hosp_1.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "하지만 직관에 반하는 결과를 얻었습니다. 어떻게 treatment가 입원 일수를 증가시킬 수 있을까요? 정답은 우리가 두 가지 다른 실험을 하고 있다는 사실에 있습니다. 중증도는 입원일수 증가와 긍정적으로 연관되며, 중증환자가 많은 병원에서 약물을 더 많이 투여하기 때문에, 약물은 입원 일수 증가와 확실히 상관 관계가 있습니다. 두 병원을 함께 살펴보면 $E[Y_0|T=0]<E[Y_0|T=1]$ 가 있습니다. 즉 중증이 아닌 환자가 있는 병원에 치료를 받지 않은 환자가 더 많기 때문에 치료를 받지 않은 환자의 잠재적 결과는 평균적으로 치료를 받은 그룹의 잠재적 결과보다 낮습니다. 즉, 중증도는 환자가 가는 병원과 약을 받을 확률을 결정하는 교란 요인으로 작용합니다.\n",
    "\n",
    "이러한 결과를 수정하는 방법은 두 가지가 있습니다. 첫 번째는 두 병원 모두 데이터를 사용하는 목적에 어긋나는 것으로 각 병원별로 ATE를 살펴보는 것입니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>   30.4074</td> <td>    2.868</td> <td>   10.602</td> <td> 0.000</td> <td>   24.523</td> <td>   36.292</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>treatment</th> <td>  -11.4074</td> <td>   10.921</td> <td>   -1.045</td> <td> 0.306</td> <td>  -33.816</td> <td>   11.001</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hosp_2 = smf.ols('days ~ treatment', data=hospital.query(\"hospital==0\")).fit()\n",
    "hosp_2.summary().tables[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>   59.0000</td> <td>    6.747</td> <td>    8.745</td> <td> 0.000</td> <td>   45.442</td> <td>   72.558</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>treatment</th> <td>  -10.3958</td> <td>    6.955</td> <td>   -1.495</td> <td> 0.141</td> <td>  -24.371</td> <td>    3.580</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hosp_3 = smf.ols('days ~ treatment', data=hospital.query(\"hospital==1\")).fit()\n",
    "hosp_3.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 경우, 우리는 직관적인 ATE를 얻었습니다. 약이 사실상 병원 입원 일수를 줄여주는 것 같습니다. 다만, 각 병원을 개별적으로 보고 있기 때문에, 데이터가 충분하지 않습니다. 따라서 통계적으로 유의한 결과를 찾을 수 없습니다.\n",
    "\n",
    "회귀의 힘을 활용하는 다른 접근법은 모델에 중증도 변수를 포함하여 통제하는 것입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>   11.6641</td> <td>    2.000</td> <td>    5.832</td> <td> 0.000</td> <td>    7.681</td> <td>   15.647</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>treatment</th> <td>   -7.5912</td> <td>    2.269</td> <td>   -3.345</td> <td> 0.001</td> <td>  -12.110</td> <td>   -3.073</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>severity</th>  <td>    2.2741</td> <td>    0.154</td> <td>   14.793</td> <td> 0.000</td> <td>    1.968</td> <td>    2.580</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hosp_4 = smf.ols('days ~ treatment + severity', data=hospital).fit()\n",
    "hosp_4.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음 질문은 병원 변수도 모형에 포함시켜야 하는가 입니다. 아무튼 병원이 치료의 원인인 것은 알고 있는 것이군요? 일단 중증도을 통제하고나면 병원 변수는 더 이상 입원 일수와 상관관계가 없습니다. 교란 요인이 되기 위해서는 treatment와 outcome에 모두 원인이 되어야 한다는 것을 알고 있습니다. 이 경우 treatment에만 원인이 되는 변수(병원 변수)가 있으며, 이것을 통제할 경우 분산이 줄어들 수도 있습니다. 그럴까요? 음, 또 사실이 아니네요. 통제요인이 반응변수의 분산을 낮추기 위해서는 병원변수처럼 treatment의 예측변수가 아니라 outcome의 예측변수가 되어야 합니다.\n",
    "\n",
    "그럼에도 우리는 이 것을 통제하고 싶겠죠? 이 것은 해로울까요? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>   11.0111</td> <td>    2.118</td> <td>    5.198</td> <td> 0.000</td> <td>    6.792</td> <td>   15.230</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>treatment</th> <td>   -5.0945</td> <td>    3.492</td> <td>   -1.459</td> <td> 0.149</td> <td>  -12.049</td> <td>    1.861</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>severity</th>  <td>    2.3865</td> <td>    0.195</td> <td>   12.251</td> <td> 0.000</td> <td>    1.999</td> <td>    2.774</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>hospital</th>  <td>   -4.1535</td> <td>    4.413</td> <td>   -0.941</td> <td> 0.350</td> <td>  -12.943</td> <td>    4.636</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hosp_5 = smf.ols('days ~ treatment + severity + hospital', data=hospital).fit()\n",
    "hosp_5.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "놀랍게도, 해로울 수 있습니다!\n",
    "\n",
    "![img](./data/img/beyond-conf/shocked.png)\n",
    "\n",
    "중증도 외에 병원을 통제변수로 추가하자 ATE 추정치에 더 많은 편차가 발생했습니다. 어떻게 이런 결과가 나왔을까요? 답은 회귀 계수의 표준오차 공식에 있습니다.\n",
    "\n",
    "$\n",
    "\\hat{\\sigma}^2 = \\dfrac{1}{n-2} \\sum( y_i - \\hat{y}_i )^2\n",
    "$\n",
    "\n",
    "$\n",
    "\\text{Var}(\\hat{\\beta}_2) = \\dfrac{\\sigma^2}{\\sum(x_i - \\bar{x})^2}\n",
    "$\n",
    "\n",
    "이 공식에서 표준오차는 X의 분산에 반비례한다는 것을 알 수 있습니다. 즉, X가 크게 변하지 않는다면, outcome에 미치는 효과를 추정하기 어렵습니다. 이것 직관적으로 이해될 수 있습니다. 극단적으로 약물의 효과를 추정하는 척하고 만 명을 대상으로 검사를 실시하지만, 그 중 한 명만 치료를 받습니다. 이것은 ATE를 찾는 것을 매우 어렵게 만들 것입니다. 우리는 한 명의 개인과 다른 모든 사람을 비교하는 것에 의존해야 할 것입니다. 이를 다르게 표현하면 치료의 효과를 쉽게 찾을 수 있도록 치료에 대한 더 많은 다양성이 필요하다는 것입니다.\n",
    "\n",
    "즉, 모형에 병원 변수를 추가하면 추정량의 오차가 증가하는 이유는 treatment에는 좋은 예측변수이지만 outcome에는 좋은 예측변수가 아니기 때문입니다(중증도를 통제할 경우). 따라서 treatment를 예측함으로써 분산을 낮추는 효과를 가질 수 있습니다! 위의 회귀모형을 두 단계로 분해하여 확인할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "        <td></td>           <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th>     <td> 4.465e-14</td> <td>    0.827</td> <td>  5.4e-14</td> <td> 1.000</td> <td>   -1.646</td> <td>    1.646</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>res_treatment</th> <td>   -5.0945</td> <td>    3.447</td> <td>   -1.478</td> <td> 0.143</td> <td>  -11.957</td> <td>    1.768</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_treatment = smf.ols('treatment ~ severity + hospital', data=hospital).fit()\n",
    "model_days = smf.ols('days ~ severity + hospital', data=hospital).fit()\n",
    "\n",
    "residuals = pd.DataFrame(dict(res_days=model_days.resid, res_treatment=model_treatment.resid))\n",
    "\n",
    "model_treatment = smf.ols('res_days ~ res_treatment', data=residuals).fit()\n",
    "\n",
    "model_treatment.summary().tables[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Treatment Variance 0.234375\n",
      "Treatment Residual Variance 0.057529091872119076\n"
     ]
    }
   ],
   "source": [
    "print(\"Treatment Variance\", np.var(hospital[\"treatment\"]))\n",
    "print(\"Treatment Residual Variance\", np.var(residuals[\"res_treatment\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "또한, 제 말을 믿지 마세요! 위의 SE 수식이 참임을 확인할 수 있습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SE of the Coeficient: 3.446973767486902\n"
     ]
    }
   ],
   "source": [
    "sigma_hat = sum(model_treatment.resid**2)/(len(model_treatment.resid)-2)\n",
    "var = sigma_hat/sum((residuals[\"res_treatment\"] - residuals[\"res_treatment\"].mean())**2)\n",
    "print(\"SE of the Coeficient:\", np.sqrt(var))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "따라서, 결론은 위의 모델의 중증도 변수와 같이 treatment 및 outcome(혼란 요인)과 관련된 통제변수를 추가해야 한다는 것입니다. 또한 교란 요인이 아니더라도 outcome에 좋은 예측변수가 되는 통제변수를 추가해야 합니다. 왜냐하면 이러한 통제변수는 추정치의 분산을 낮추기 때문입니다. 그러나 treatment에 좋은 예측변수인 통제변수는 추정량의 분산을 증가시키므로 추가해서는 안됩니다.\n",
    "\n",
    "이 상황을 인과관계 그래프에서 어떻게 나타낼 수 있는지 확인해봅시다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n -->\n<!-- Pages: 1 -->\n<svg width=\"199pt\" height=\"260pt\"\n viewBox=\"0.00 0.00 199.35 260.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 256)\">\n<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-256 195.35,-256 195.35,4 -4,4\"/>\n<!-- X -->\n<g id=\"node1\" class=\"node\">\n<title>X</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"27\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"27\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">X</text>\n</g>\n<!-- T -->\n<g id=\"node2\" class=\"node\">\n<title>T</title>\n<ellipse fill=\"none\" stroke=\"gold\" cx=\"27\" cy=\"-162\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"27\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\">T</text>\n</g>\n<!-- X&#45;&gt;T -->\n<g id=\"edge1\" class=\"edge\">\n<title>X&#45;&gt;T</title>\n<path fill=\"none\" stroke=\"black\" d=\"M27,-215.7C27,-207.98 27,-198.71 27,-190.11\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"30.5,-190.1 27,-180.1 23.5,-190.1 30.5,-190.1\"/>\n</g>\n<!-- Y -->\n<g id=\"node3\" class=\"node\">\n<title>Y</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"27\" cy=\"-90\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"27\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">Y</text>\n</g>\n<!-- T&#45;&gt;Y -->\n<g id=\"edge2\" class=\"edge\">\n<title>T&#45;&gt;Y</title>\n<path fill=\"none\" stroke=\"black\" d=\"M27,-143.7C27,-135.98 27,-126.71 27,-118.11\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"30.5,-118.1 27,-108.1 23.5,-118.1 30.5,-118.1\"/>\n</g>\n<!-- treatment -->\n<g id=\"node4\" class=\"node\">\n<title>treatment</title>\n<ellipse fill=\"none\" stroke=\"gold\" cx=\"116\" cy=\"-90\" rx=\"44.39\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"116\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">treatment</text>\n</g>\n<!-- days -->\n<g id=\"node7\" class=\"node\">\n<title>days</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"152\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"152\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">days</text>\n</g>\n<!-- treatment&#45;&gt;days -->\n<g id=\"edge6\" class=\"edge\">\n<title>treatment&#45;&gt;days</title>\n<path fill=\"none\" stroke=\"black\" d=\"M124.71,-72.05C128.98,-63.77 134.2,-53.62 138.92,-44.42\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"142.15,-45.8 143.61,-35.31 135.93,-42.6 142.15,-45.8\"/>\n</g>\n<!-- severity -->\n<g id=\"node5\" class=\"node\">\n<title>severity</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"153\" cy=\"-234\" rx=\"38.19\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"153\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">severity</text>\n</g>\n<!-- hospital -->\n<g id=\"node6\" class=\"node\">\n<title>hospital</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"119\" cy=\"-162\" rx=\"38.19\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"119\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\">hospital</text>\n</g>\n<!-- severity&#45;&gt;hospital -->\n<g id=\"edge3\" class=\"edge\">\n<title>severity&#45;&gt;hospital</title>\n<path fill=\"none\" stroke=\"black\" d=\"M144.77,-216.05C140.8,-207.89 135.96,-197.91 131.54,-188.82\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"134.67,-187.25 127.15,-179.79 128.38,-190.31 134.67,-187.25\"/>\n</g>\n<!-- severity&#45;&gt;days -->\n<g id=\"edge4\" class=\"edge\">\n<title>severity&#45;&gt;days</title>\n<path fill=\"none\" stroke=\"black\" d=\"M158.09,-215.98C160.94,-205.67 164.27,-192.19 166,-180 172.75,-132.46 176.42,-119.44 169,-72 167.62,-63.18 165.02,-53.82 162.26,-45.46\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"165.51,-44.13 158.89,-35.86 158.9,-46.45 165.51,-44.13\"/>\n</g>\n<!-- hospital&#45;&gt;treatment -->\n<g id=\"edge5\" class=\"edge\">\n<title>hospital&#45;&gt;treatment</title>\n<path fill=\"none\" stroke=\"black\" d=\"M118.26,-143.7C117.93,-135.98 117.53,-126.71 117.16,-118.11\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"120.66,-117.95 116.73,-108.1 113.66,-118.25 120.66,-117.95\"/>\n</g>\n</g>\n</svg>\n",
      "text/plain": [
       "<graphviz.dot.Digraph at 0x12ccb7550>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g = gr.Digraph()\n",
    "\n",
    "g.edge(\"X\", \"T\"), g.edge(\"T\", \"Y\")\n",
    "g.node(\"T\", color=\"gold\")\n",
    "\n",
    "g.node(\"treatment\", color=\"gold\")\n",
    "g.edge(\"severity\", \"hospital\")\n",
    "g.edge(\"severity\", \"days\")\n",
    "g.edge(\"hospital\", \"treatment\")\n",
    "g.edge(\"treatment\", \"days\")\n",
    "\n",
    "g"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bad Controls - Selection Bias\n",
    "\n",
    "이메일 예시로 돌아가보겠습니다. 이메일은 무작위로 고객에게 할당되었다는 것을 기억하세요. 우리는 이미 `credit_limit`, `risk_score`이 무엇인지 설명했습니다. 이제 남은 변수들을 살펴봅시다. `opened`는 이메일을 열었는지를 나타내는 더미 변수입니다. `agreement`는 이메일을 수신한 후에 관련 부서에 연락하여 채무협상을 한 경우를 나타내는 더미변수입니다. 다음 모델 중 어떤 모델이 더 적절하다고 생각하십니까? 첫 번째 모형은 treatment(email) + credit_limit + risk_score이고, 두 번째 모형은 첫 번째 모형에 `opened`, `agreement` 변수를 추가한 모형입니다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "        <td></td>          <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th>    <td>  490.8653</td> <td>    9.715</td> <td>   50.527</td> <td> 0.000</td> <td>  471.820</td> <td>  509.911</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>email</th>        <td>    4.4304</td> <td>    2.130</td> <td>    2.080</td> <td> 0.038</td> <td>    0.255</td> <td>    8.606</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>credit_limit</th> <td>    0.1511</td> <td>    0.008</td> <td>   18.833</td> <td> 0.000</td> <td>    0.135</td> <td>    0.167</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>risk_score</th>   <td>   -8.0516</td> <td>   38.424</td> <td>   -0.210</td> <td> 0.834</td> <td>  -83.379</td> <td>   67.276</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "email_1 = smf.ols('payments ~ email + credit_limit + risk_score', data=data).fit()\n",
    "email_1.summary().tables[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "        <td></td>          <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th>    <td>  488.4416</td> <td>    9.716</td> <td>   50.272</td> <td> 0.000</td> <td>  469.394</td> <td>  507.489</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>email</th>        <td>   -1.6095</td> <td>    2.724</td> <td>   -0.591</td> <td> 0.555</td> <td>   -6.949</td> <td>    3.730</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>credit_limit</th> <td>    0.1507</td> <td>    0.008</td> <td>   18.809</td> <td> 0.000</td> <td>    0.135</td> <td>    0.166</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>risk_score</th>   <td>   -2.0929</td> <td>   38.375</td> <td>   -0.055</td> <td> 0.957</td> <td>  -77.325</td> <td>   73.139</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>opened</th>       <td>    3.9808</td> <td>    3.914</td> <td>    1.017</td> <td> 0.309</td> <td>   -3.692</td> <td>   11.654</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>agreement</th>    <td>   11.7093</td> <td>    4.166</td> <td>    2.811</td> <td> 0.005</td> <td>    3.542</td> <td>   19.876</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.table.SimpleTable'>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "email_2 = smf.ols('payments ~ email + credit_limit + risk_score + opened + agreement', data=data).fit()\n",
    "email_2.summary().tables[1]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "첫 번째 모형에서 이메일 변수는 통계적으로 유의해보이지만, 두 번째 모형에서는 그렇지 않습니다. 하지만 실제로는 두 번째 모형이 옳은 모형일 수도 있고, treatment(이메일)은 outcome(지불금액)에 효과가 없을 수도 있습니다. 결국 이 모형은 더 많은 요소들을 통제하기 때문에 더 로버스트할까요? 지금쯤 그렇지 않다는 것을 알고 있을 것입니다. 남은 것은 이것이 무엇인지 알아내는 것입니다.\n",
    "\n",
    "우리는 treatment와 outcome에 모두 원인이 되는 교란 요인을 변수로 추가해야 한다는 것을 알았습니다. 또한 outcome을 매우 잘 예측하는 통제변수를 추가하는 것도 좋은 아이디어라는 것을 알고 있습니다. 이것은 필수는 아니지만, 추가하면 좋습니다. 또한 treatment만을 예측하는 통제변수를 추가하는 것은 좋지 않은 생각이라는 것도 알고 있습니다. 다시 말하지만, 이것은 치명적인 것은 아니지만 피하는 것이 좋습니다. 그럼 `opened`, `agreement`는 어떤 종류의 통제변수일까요? 이 변수는 위에서 설명한 것들에 해당하지 않습니다.\n",
    "\n",
    "직관적으로 생각해보면 `opened`, `agreement`는 이메일과 확실히 관계가 있습니다. 이메일을 받지 못한 경우에는 이메일을 열 수 없습니다. 또한 agreement도 이메일을 수신한 후에 연락한 경우만 고려한다고 했습니다. **이 변수들은 이메일 변수의 원인은 아닙니다! 대신, 이메일 변수가 이 변수들의 원인입니다!**\n",
    "\n",
    "어떤 변수를 다루고 있는지 이해해야 할 때마다, 항상 그것들의 인과 그래프에 대해 생각합시다. 여기서 해보죠."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n -->\n<!-- Pages: 1 -->\n<svg width=\"303pt\" height=\"260pt\"\n viewBox=\"0.00 0.00 303.34 260.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 256)\">\n<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-256 299.34,-256 299.34,4 -4,4\"/>\n<!-- email -->\n<g id=\"node1\" class=\"node\">\n<title>email</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"30.55\" cy=\"-234\" rx=\"30.59\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"30.55\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">email</text>\n</g>\n<!-- payments -->\n<g id=\"node2\" class=\"node\">\n<title>payments</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"170.55\" cy=\"-18\" rx=\"44.69\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"170.55\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">payments</text>\n</g>\n<!-- email&#45;&gt;payments -->\n<g id=\"edge1\" class=\"edge\">\n<title>email&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M25.81,-216.06C18.06,-184.46 6.44,-115.8 37.55,-72 56.02,-45.99 89.41,-32.73 118.11,-25.99\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"118.99,-29.38 128.03,-23.86 117.52,-22.53 118.99,-29.38\"/>\n</g>\n<!-- opened -->\n<g id=\"node3\" class=\"node\">\n<title>opened</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"132.55\" cy=\"-162\" rx=\"36.29\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"132.55\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\">opened</text>\n</g>\n<!-- email&#45;&gt;opened -->\n<g id=\"edge2\" class=\"edge\">\n<title>email&#45;&gt;opened</title>\n<path fill=\"none\" stroke=\"black\" d=\"M49.73,-219.83C64.87,-209.45 86.23,-194.79 103.47,-182.96\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"105.78,-185.61 112.05,-177.07 101.82,-179.84 105.78,-185.61\"/>\n</g>\n<!-- agreement -->\n<g id=\"node4\" class=\"node\">\n<title>agreement</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"94.55\" cy=\"-90\" rx=\"48.19\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"94.55\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">agreement</text>\n</g>\n<!-- email&#45;&gt;agreement -->\n<g id=\"edge3\" class=\"edge\">\n<title>email&#45;&gt;agreement</title>\n<path fill=\"none\" stroke=\"black\" d=\"M38.01,-216.43C49.03,-191.98 69.76,-146 82.82,-117.02\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"86.01,-118.46 86.93,-107.9 79.63,-115.58 86.01,-118.46\"/>\n</g>\n<!-- opened&#45;&gt;payments -->\n<g id=\"edge4\" class=\"edge\">\n<title>opened&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M139.3,-144C143.22,-133.7 148.07,-120.22 151.55,-108 157.35,-87.61 162.29,-64.15 165.68,-46.44\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"169.19,-46.7 167.58,-36.23 162.3,-45.42 169.19,-46.7\"/>\n</g>\n<!-- opened&#45;&gt;agreement -->\n<g id=\"edge5\" class=\"edge\">\n<title>opened&#45;&gt;agreement</title>\n<path fill=\"none\" stroke=\"black\" d=\"M123.54,-144.41C119.06,-136.16 113.55,-126.01 108.54,-116.78\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"111.6,-115.07 103.75,-107.96 105.45,-118.41 111.6,-115.07\"/>\n</g>\n<!-- agreement&#45;&gt;payments -->\n<g id=\"edge6\" class=\"edge\">\n<title>agreement&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M111.79,-73.12C121.9,-63.81 134.83,-51.9 145.99,-41.62\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"148.39,-44.17 153.37,-34.82 143.64,-39.02 148.39,-44.17\"/>\n</g>\n<!-- credit_limit -->\n<g id=\"node5\" class=\"node\">\n<title>credit_limit</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"131.55\" cy=\"-234\" rx=\"51.99\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"131.55\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">credit_limit</text>\n</g>\n<!-- credit_limit&#45;&gt;payments -->\n<g id=\"edge7\" class=\"edge\">\n<title>credit_limit&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M150.93,-217C160.95,-207.45 172.18,-194.38 177.55,-180 194.24,-135.3 185.59,-78.71 177.95,-45.9\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"181.33,-45 175.54,-36.13 174.53,-46.68 181.33,-45\"/>\n</g>\n<!-- credit_limit&#45;&gt;opened -->\n<g id=\"edge8\" class=\"edge\">\n<title>credit_limit&#45;&gt;opened</title>\n<path fill=\"none\" stroke=\"black\" d=\"M131.79,-215.7C131.9,-207.98 132.04,-198.71 132.16,-190.11\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"135.66,-190.15 132.3,-180.1 128.66,-190.05 135.66,-190.15\"/>\n</g>\n<!-- credit_limit&#45;&gt;agreement -->\n<g id=\"edge9\" class=\"edge\">\n<title>credit_limit&#45;&gt;agreement</title>\n<path fill=\"none\" stroke=\"black\" d=\"M112.99,-216.84C103.4,-207.23 92.66,-194.17 87.55,-180 80.33,-160.01 82.85,-135.91 86.75,-117.83\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"90.19,-118.5 89.14,-107.96 83.39,-116.85 90.19,-118.5\"/>\n</g>\n<!-- risk_score -->\n<g id=\"node6\" class=\"node\">\n<title>risk_score</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"248.55\" cy=\"-234\" rx=\"46.59\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"248.55\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">risk_score</text>\n</g>\n<!-- risk_score&#45;&gt;payments -->\n<g id=\"edge10\" class=\"edge\">\n<title>risk_score&#45;&gt;payments</title>\n<path fill=\"none\" stroke=\"black\" d=\"M244.52,-215.67C237.28,-185.73 220.84,-122.63 199.55,-72 195.67,-62.77 190.65,-53.05 185.88,-44.5\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"188.87,-42.68 180.87,-35.74 182.8,-46.16 188.87,-42.68\"/>\n</g>\n<!-- risk_score&#45;&gt;opened -->\n<g id=\"edge11\" class=\"edge\">\n<title>risk_score&#45;&gt;opened</title>\n<path fill=\"none\" stroke=\"black\" d=\"M224.51,-218.5C206.87,-207.85 182.75,-193.29 163.64,-181.76\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"165.35,-178.71 154.98,-176.54 161.73,-184.7 165.35,-178.71\"/>\n</g>\n<!-- risk_score&#45;&gt;agreement -->\n<g id=\"edge12\" class=\"edge\">\n<title>risk_score&#45;&gt;agreement</title>\n<path fill=\"none\" stroke=\"black\" d=\"M237.67,-216.22C224.82,-197.23 202.08,-166.08 177.55,-144 163.31,-131.19 145.64,-119.44 130.27,-110.28\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"131.93,-107.2 121.53,-105.2 128.41,-113.25 131.93,-107.2\"/>\n</g>\n</g>\n</svg>\n",
      "text/plain": [
       "<graphviz.dot.Digraph at 0x12ccda9e8>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g = gr.Digraph()\n",
    "\n",
    "g.edge(\"email\", \"payments\")\n",
    "g.edge(\"email\", \"opened\")\n",
    "g.edge(\"email\", \"agreement\")\n",
    "g.edge(\"opened\", \"payments\")\n",
    "g.edge(\"opened\", \"agreement\")\n",
    "g.edge(\"agreement\", \"payments\")\n",
    "\n",
    "g.edge(\"credit_limit\", \"payments\")\n",
    "g.edge(\"credit_limit\", \"opened\")\n",
    "g.edge(\"credit_limit\", \"agreement\")\n",
    "g.edge(\"risk_score\", \"payments\")\n",
    "g.edge(\"risk_score\", \"opened\")\n",
    "g.edge(\"risk_score\", \"agreement\")\n",
    "\n",
    "g"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이메일은 랜덤으로 설계되어 있기 때문에 원인이 되는 것은 아무것도 없습니다. 그리고 신용한도와 리스크가 지불금액의 원인이 된다는 것을 알고 있습니다(또는 적어도 믿을 만한 강한 근거가 있습니다). 또한 우리는 이메일이 지불금액의 원인이 된다고 생각합니다. `opened`는 지불금액의 원인이 된다고 생각합니다. 직관적으로 이메일을 연 사람들은 더 기꺼이 연락하고 빚을 갚을 것입니다. `opened`가 지불금액의 원인이 되는 것과 같은 이유로 agreement의 원인이 된다고 생각합니다. 게다가, 이메일은 `opened`의 원인이 된다고 알고 있으며, 다른 리스크와 신용한도를 가진 사람들이 이메일에 대해 다른 open rate를 갖는다는 믿을 만한 이유가 있으므로, 리스크와 신용한도는 `opened`의 원인이 됩니다. 또한 agreement에 관해서도, `opened`가 원인이 될 수 있다고 생각됩니다. 지불금액에 대해 생각해 보면, 다음과 같은 퍼널의 결과를 생각할 수 있습니다.\n",
    "\n",
    "$\n",
    "email -> opened -> agreement -> payment \n",
    "$\n",
    "\n",
    "또한 다른 수준의 리스크를 가진 경우 agreement를 수행하는 성향이 다를 것으로 생각되기 때문에 리스크를 agreement의 원인으로 표시했습니다. 이메일과 agreement에 관해서, 어떤 사람들은 이메일의 제목을 읽기만 하면 agreement할 가능성이 높아진다고 주장할 수 있습니다. 요점은 메일을 열지 않고, agreement를 유발할 수도 있다는 것입니다. \n",
    "\n",
    "이 그래프에서 알 수 있는 것은 이메일에서 지불금액까지의 인과경로에 opened와 agreement가 모두 있다는 것입니다. 따라서 회귀모형으로 이것들을 통제한다면 \"`opened` 및 `agreement`가 고정되었을 때의 이메일의 효과입니다.\"라고 말할 것입니다. 그러나 둘 다 이메일의 인과 효과의 일부분이기 때문에 고정하고 싶지 않습니다. 대신, 이메일은 agreement의 비율을 높이기 때문에 정확하게 지불금액을 증가시킨다고 주장할 수 있습니다. 이러한 변수를 고정하면 이메일의 실제 효과 중 일부가 제거될 수 있습니다.\n",
    "\n",
    "\n",
    "잠재적 결과 표기법에서는 무작위화로 인해 $E[Y_0|T=0] = E[Y_0|T=1]$라고 표기할 수 있습니다. 그러나 무작위화를 사용하더라도 agreement를 통제할 경우, treatment와 control를 더 이상 비교할 수 없습니다. 사실, 직관적인 사고를 통해 이 것들이 어떻게 다른지 추측할 수 있습니다:\n",
    "\n",
    "$\n",
    "E[Y_0|T=0, Agreement=0] > E[Y_0|T=1, Agreement=0]\n",
    "$\n",
    "\n",
    "$\n",
    "E[Y_0|T=0, Agreement=1] > E[Y_0|T=1, Agreement=1]\n",
    "$\n",
    "\n",
    "첫 번째 식은 이메일(빛를 갚을 것을 요청하는 이메일을 수신했는지 유무)과 agreement(이메일을 수신한 후에 관련 부서에 연락하여 채무협상을 한 경우)가 없는 사람이 이메일이 있고 agreement가 없는 사람보다 지불금액이 더 크다라는 것을 분명히 합니다. treatment가 긍정적인 효과가 있다면 **이메일을 받고도** agreement가 없는 사람은 agreement가 없는 사람에 비해 지불 측면에서 더 나쁠 것입니다. 두 번째 식은 이메일이 없고, agreement가 있는 사람이 이메일이 있고 agreement도 있는 사람에 비해 더 나을 수도 있을 것입니다.\n",
    "\n",
    "처음 읽었을 때 매우 혼란스러울 수 있지만(저에게는 해당), 이해하시기 바랍니다. 필요한 경우 다시 읽어 보십시오. 그러면 opened에 대해서도 비슷한 추론을 할 수 있습니다. 직접 만들어 보세요.\n",
    "\n",
    "이런 종류의 편향은 너무 만연해서 나름의 명칭을 가지고 있습니다. 교란요인은 공통 원인에 대한 통제 실패에서 오는 편향이지만, **선택 편향은 원인에서 효과까지의 경로 사이에 있는 공통   효과 또는 변수에 대해 통제할 때 발생합니다**. 경험적으로 항상 모형에 Y를 잘 예측할 수 있는 변수 혹은 교란 요인을 항상 포함시켜야 합니다. Treatment만을 잘 예측하는 변수 혹은 treatment와 outcome 사이의 매개변수 또는 treatment와 outcome의 공통 효과인 변수는 항상 제외하십시오.\n",
    "\n",
    "![img](./data/img/beyond-conf/selection.png)\n",
    "\n",
    "선택 편향은 너무 만연해서 랜덤화조차 해결할 수 없습니다. 더 나은건, 임의의 데이터에서도 잘못된 조언에 의해 종종 도입된다는 것입니다! 선택 편향을 발견하고 피하려면 기술보다 더 많은 연습이 필요합니다. 종종, 그것들은 아마도 영리한 아이디어 아래에서 나타나서, 발견하는 것을 더욱 어렵게 만듭니다. 다음은 선택 편향의 몇 가지 예입니다.\n",
    "\n",
    "    1. Adding a dummy for paying the entire debt when trying to estimate the effect of a collections strategy on payments.\n",
    "    2. Controlling for white vs blue collar jobs when trying to estimate the effect of schooling on earnings\n",
    "    3. Controlling for conversion when estimating the impact of interest rates on loan duration\n",
    "    4. Controlling for marital happiness when estimating the impact of children on extramarital affairs\n",
    "    5. Breaking up payments modeling E[Payments] into one binary model that predict if payment will happen and another model that predict how much payment will happen given that some will: E[Payments|Payments>0]*P(Payments>0)\n",
    "    \n",
    "이 모든 아이디어에서 주목할 만한 것은 그것이 얼마나 합리적으로 들리는가에 대한 것입니다. 선택편향은 종종 그러합니다. 이것을 경고로 삼으세요. 사실, 선택편향이 얼마나 나쁜지 알기 전에 여러 번 위의 함정에 빠졌었습니다. 특히 마지막 한 가지는 매우 영리해 보이고 많은 데이터 과학자들의 허를 찌르기 때문에 더 자세히 설명할 가치가 있습니다. 너무 널리 퍼져 있어서, 독자적인 이름을 가지고 있습니다. **The Bad COP**!\n",
    "\n",
    "### Bad COP\n",
    "\n",
    "상황은 이렇습니다. 예측하려는 연속형 변수가 있지만 분포가 0이 매우 많습니다. 예를 들어, 고객의 지출 분포를 모형화하려면 감마 분포와 비슷하지만 0이 많습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaQAAAE0CAYAAABjKkd2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7oklEQVR4nO3deVxV1cL/8c9x6iqoGBwgBzQFccjZADUnHIi4iYimWZaW4tCgZjg0WZYXp4tThBpOlZaGXoc0zAGnVDBvhpX5Q01TS0DyOD2aJef3hw/78QQKCsJWv+/Xi1e599p7r7U4nO/Za+19tsVms9kREREpZiWKuwIiIiKgQBIREZNQIImIiCkokERExBQUSCIiYgoKJBERMQUFUjGLiorCxcWFbdu2Fcvxt23bhouLC1FRUQ7LQ0JCcHFx4ejRo8VSL4CjR4/i4uLC4MGDi60OImbm4uJCSEhIcVej0CiQCoGLi4vDj9VqpWbNmrRq1YpBgwaxevVq/vzzz9ty7MGDBxdroBVEduDcqX9QJ06cYNy4cbRv354aNWrg5uZGzZo1+ec//8nMmTP5/fffi6wuZvgAYSZnzpwhKiqKNm3aULVqVdzd3alTpw6BgYGMGjWK5OTk4q6i5KJUcVfgbjJq1CgArly5wtmzZ0lNTWXlypV89tln+Pj4MGfOHJo0aeKwTUREBOHh4VStWrU4qkyzZs1ITk7G1dW1WI5/I5UrVyY5OZkKFSoUd1Vy+Oijj4iMjOSPP/6gXr16dOvWjUqVKnH69GmSk5N58803mTx5Mvv378fJyam4q3tPOXnyJI8++ihHjhyhevXqhIeH4+bmxpkzZ0hJSSEuLo7Lly/j5+dX3FWVv1EgFaIxY8bkWGaz2Xj33XeZO3cuXbt2ZePGjXh7exvrXV1dizUMypUrR+3atYvt+DdSunRpU9Zt6dKlvPzyy1SsWJEFCxYQHByco8yePXsYNWrUbTszluuLioriyJEjPP3008ycOROLxeKwPjMzk59//rmYaic3oiG728zFxYV///vfPPHEE5w5c4axY8c6rL/eHNKOHTvo1asX9evXx93dHW9vb9q1a8frr7+O3X71254aNGjAp59+CsDjjz/uMGyY7dohvc8++4z27dtTuXJlHnnkEeD6c0jZ7HY7M2fOpHnz5nh4eFC/fn1ef/11zp07l2tbrzf89vd2Llq0iEaNGgHw9ddfO9Q9uy43mkNKT09n5MiRNGrUCHd3dx588EF69uzJ119/naNsdhsHDx7M0aNHee6556hZsyYeHh60a9eOhISEXOucm3PnzhlnwnFxcbmGEVw980xISDDO7vKaD8v+Pf19yO3LL78kNDSUOnXq4O7ujq+vL0FBQfz73/82yri4uBjtbtSokdGPDRo0cNjXkSNHGDJkCPXq1cNqteLj40Pfvn35/vvvc9Rn0aJFxu/i22+/JTw8HC8vL6pXr06fPn04fvy4sc/nn38eb29vPD09CQkJYd++fbm28dKlS8ycOZO2bdtSpUoVKleuTLt27Zg3b57xms527XDub7/9xgsvvICvry/3338/X3zxRa77z5aUlARcHX34exjB1Q+BzZs3d1iW/fpctGgRX375JZ06daJy5crUqFGDvn37XjfAbrVNmZmZDB06FF9fX9zd3QkICOCTTz7J9RiXL19m0qRJNG7cGHd3dxo2bMh7773HH3/8ccN+uBPpDKmIjBkzhqVLl5KQkMDZs2dvOAy1YcMGnnjiCZydnQkODqZKlSrYbDYOHTrE7NmzeeeddyhVqhSDBw9m8eLFfP/99zz55JN4eXldd5/vv/8+W7ZsITg4mLZt23L58uV81Xv06NHs2rWLsLAwKlSowPr164mJiWHXrl2sXbuW++6776b7Aq6G6aBBg5g1axbVqlWjd+/exrrssLyeX375heDgYE6cOEGrVq3o1q0bJ0+eZMWKFaxfv56ZM2fy1FNP5dju2LFjdOjQgRo1atCzZ09Onz7Nf/7zH3r37s2KFSto06ZNnvVeuXIlp0+fpnnz5nTq1OmGZUuVKtif14IFCxg2bBju7u4EBQVhtVrJzMzkwIEDzJ8/nxEjRgBXh4oXL17MsWPHGDRoEBUrVgQw/guwd+9eQkNDOXv2LJ07d6Z+/fr8/PPPrF69moSEBBYvXkxgYGCOOnz77bfMmDGDNm3a8Mwzz7Bnzx5Wr17N/v37Wbx4MY8++ij16tWjV69eHDhwgPXr1xMWFsbevXtxdnY29nPu3Dm6du3Knj17aNiwofH73rhxI6+88gq7d+8mNjY2x/FPnz5Np06dqFChAqGhodjtdipVqnTDfstef+jQIRo2bHhTfb569Wo2bNjA448/TuvWrUlJSWHFihVs27aNr776ilq1ahW4TWfOnCEoKIgyZcrQpUsXLl++zIoVK3jxxRcpUaKEw9+C3W6nb9++rF27lho1ajBgwAD+/PNPFi1axA8//HBTbbsTKJCKyIMPPkiVKlU4ceIEe/fuveGb38KFC8nKyuKLL77I8Qf1+++/G290Q4YMYd++fXz//ff07t2b1q1bX3ef2X9QN/sHmpyczLZt26hWrRoAb731Fn369GHt2rXExMTwyiuv3NT+sjVs2JCKFSsya9YsvLy8ch3uvJ5XXnmFEydOMHr0aEaPHm0sf/HFF+nYsSOvvPIK7dq1o0qVKg7bbd++Pcc2PXr0IDw8nJkzZ+YrkHbt2gVAu3bt8l3fWzV//nzKlCnD9u3bcXd3d1iXmZlp/P+YMWPYvn07x44dY/DgwVSvXt2hrN1uZ9CgQZw5c4YPPvjA4Q1v8+bNhIWFERERQUpKCuXKlXPY9quvvmLhwoWEhoYa++revTsbN26kc+fOjBgxgpdeeskoP3ToUBYuXMjHH3/scDb42muvsWfPHt5++22GDRtmLP/jjz/o06cPn376KV26dMlxxvnjjz/Ss2dPYmJi8h3wYWFh7Ny5k5dffpn//ve/tG/fnoYNG+Lm5pbntgkJCSxZsoSgoCBj2cyZM3nzzTeJjIxk+fLlBW7T999/T58+fZg2bRolS5YErp4ht2rViunTpzv8fuLj41m7di1NmzZlzZo1lC1b1jh2hw4d8tUfdxIN2RWhBx54AHB8M7mR7Bffte6///5bOvazzz5702EEMGjQICOMAEqWLMk777yDxWK57hDD7fTrr7+yYcMGqlSpkiMM69evz3PPPccff/zBkiVLcmxbrVo1IiMjHZZ16NCBqlWrsmfPnnwdPy0tDbh6wUVRKFWqFKVLl86x/GbmHZOSkvjpp59o2rSpw5sdXA3Wf/7zn5w6dYq1a9fm2LZVq1ZGGAFYLBZ69OgBXB0qfOGFFxzKP/HEEwAOw3anT5/m008/pWHDhg5v3AD33Xcfb731FkCuv7MyZcrw3nvv3dTZ5oABA3jllVf4888/mTlzJt26dcPb25v69eszcOBAY0gvN23atHEII7gaFlWrVmXTpk38+uuvBW5TuXLlGD9+vBFGAHXq1MHf358DBw5w/vx5Y/miRYsAePPNNx3eD1xcXHj11Vfz2SN3Dp0hFaHsMeXcxrWv1aNHD1avXk2HDh0ICwujdevWPPzwwzk++d6MZs2a3dJ2rVq1yrHMx8cHd3d3Dh8+zLlz5yhfvvwt1+tmfffddwD4+/tTpkyZHOvbtWtHTEyMUe5aDRo0cHgTyFa1alVTXgbco0cP3njjDfz9/QkLC6Nly5b4+/vj6el5U/vJ7ovrnQG2a9eO1atX891339G9e3eHdbl9iMk+fv369SlRwvEzbfaHruw3brh6gcdff/1FiRIlcp2r/OuvvwD4f//v/+VY5+XlhdVqvW7bcmOxWHjrrbd4+eWX2bRpE7t37+a7775jz549LFmyhCVLluQ4U86W2+u9VKlS+Pv7c/z4cVJSUqhcuXKB2lSzZs1ch+yzr7S12WzGcOd3332HxWKhZcuW+arrnU6BVIROnjwJkOfQQZcuXVi6dCkxMTF8+umnLFy4EIB69eoxatQoh0+s+fX3IZ+Cbme1WklLSyvyQDp79uwN6+Xh4QFcHaf/u2vnVK5VsmRJsrKy8nX87P1f+4Z7u7z44otYrVbmzZtHXFwcs2fPBuDhhx/mrbfeuuEQ7bUK0me5vXFmh/qN1l17dWH2/Vh79+5l7969163ntWcG2W71dQtXzyK6detGt27dALhw4QJTp05lypQpTJgwgcceeyxH4N7o9Q7/15cFadONXodw9baRbNnzzbnN1Rakb8xKQ3ZF5NChQ5w4cYJSpUrRuHHjPMt37tyZlStXcvToUdasWcPw4cM5duwYffv2Zfv27Td9/LzOyq4nPT091+UZGRkADmFksVgc/piuldub3a3IfhO8Xr2yh9Ru171LAQEBAGzZsuWmtss+k7jZ/unZsyfr1q3j559/Zvny5QwYMIDvvvuOHj16cPDgwXwdu7j7LHu/ERER2Gy26/6kpKTk2PZWX7e5cXJy4o033qBFixYAbN26NUeZvF7v2W0pSJtuRoUKFTh79myuV9Rdr653MgVSEZk4cSIAjz32mMPVR3kpW7YsrVq1YuzYsbz77rvY7XbWrFljrM/+VJXfT/g3K7fLqFNTU0lPT6dmzZoOgeTi4mJcDvx33377bY5lt1L37E+0SUlJuV4pmB0U+Qn9WxEaGkqlSpXYvXs3GzduvGHZv/76y2hb9qX4ufXPX3/9lecbV4UKFQgMDGTy5Mm8+OKLXLp0ifXr1xvrb9SX2ZfXX+/bPG53nzVv3pwSJUqwc+fO27L/m5X99/f3y7Ih99f7X3/9Zcw7Zb/+iqpNjRo1wm63s2PHjnzV9U6nQLrNbDYbI0aMYOnSpVSqVIm33347z222b99ujEFfK/uT7LVXQmVf5HDs2LHCqfDfzJo1y2HfV65cYezYsdjt9hyXVjdv3pzjx4/z1VdfOSxfuHBhrhPJLi4uWCyW64ZYbqpUqUKHDh04ceIE06dPd1i3f/9+5s2bx3333WdMrhe28uXLGx8unn/++RxtzbZ3716Cg4ONIZ7y5cvj6+tLUlKSw+W6drudCRMm5NoHW7ZsyfVN82ZfB/7+/vj6+hpzKH8/xurVq3F1deWxxx67YdtvlZubGz179mTfvn1ERUXl+to+ceJErvMtt2LGjBns378/13U7d+40gjm3OZitW7eybt06h2WxsbEcP37cuIcPiq5N2X9j7777LhcvXjSW22w2pkyZUqB9m5HmkApR9uRmVlaW8dVBO3fu5OLFi/j6+jJ79mxq1qyZ535Gjx7NiRMnCAgIwMvLi3/84x/88MMPbNy4kfvvv59nn33WKNu+fXtmzJjBuHHj2L9/v/FJ/O9Xk90qPz8/Wrdu7XAf0o8//kjTpk158cUXHcq+9NJLbNy4kaeffpquXbtitVqNMfagoKAcf+jOzs74+fmRlJREz549adSoEaVLl6Zly5Y3nLCNjo7m0UcfZfz48WzdupWHH37YuA/p0qVLTJs27bZ+FdMTTzzBpUuXiIyM5IknnqB+/fq0aNECFxcXTp8+zTfffMN3331HxYoVHa6QGzp0KEOGDCE4OJiuXbtSrlw5kpKSOHHiBI888kiOodg+ffrg5ORE8+bN8fLywmKxsGfPHnbu3MmDDz5I165djbLt27dnxYoVDB06lC5duuDs7EzFihWNm0NjY2Pp2rUrgwYN4j//+Y9xH9KqVasoU6YMs2bNynHJd2GaNGkShw8fZuLEiSxZsoSWLVvi4eFBWloaBw8eZPfu3YwfP75Qvplj6dKlvPXWW9SuXZvmzZvj6enJhQsX+Omnn9i6dSt2u53BgwfTtGnTHNsGBQXx1FNP0aVLF2rUqEFKSgobNmzg/vvvzxEARdGm7t27s3z5cr788ktatGhBSEgIf/75J6tXr6Zx48YcOnTolvdtRgqkQpT9ybl06dI4OztTuXJlQkNDCQkJ4dFHH8318t3cjBgxgjVr1vDtt98an+YqV67M4MGDGTJkiMObbfv27ZkwYQILFiwgLi7OGGsurECaMGECq1at4qOPPuKXX37Bzc2NIUOGMGbMmBwTrW3atOHTTz9l0qRJxhtdy5YtWb9+PStXrswRSACzZ8/m9ddfZ+fOnaxfv56srCxGjRp1w0CqXr06mzdvZsqUKSQkJLBr1y6cnJxo1aoVL7/8cr4n+wvimWeeoUOHDsTFxZGYmMjnn3/OhQsXKF++PHXq1GHcuHFGoGTLvuT6/fff57PPPsPZ2ZnAwEA+/vhjxo8fn+MYb7/9Nps2bWLfvn1s3LiRUqVKUbVqVUaNGsXAgQMdJsezvz0hPj6eDz74gD///JNq1aoREREBQNOmTdm8eTOTJ09m8+bNbNy4kYoVKxISEsKIESNu6ZaAm1G+fHm++OILPv74Yz7//HO++OILLl26hNVqpXr16owdO5awsLBCOdYHH3zA+vXr2bp1K19//TXp6elkZWXh7u7O448/Tp8+fa57U3OXLl149tlniY6OJiEhgdKlSxMaGsrYsWNzfJgsijZZLBYWLlzI1KlTWbx4MR9++CEeHh707t2bkSNHGhek3C0sNpst55iAiMg9JCoqiokTJxITE5Prt3xI0dAckoiImIICSURETEGBJCIipqA5JBERMQWdIYmIiCkokERExBQUSCIiYgp3bSClpqYWdxXuGerroqO+Lhrq5+Jx1waSiIjcWRRIIiJiCgokERExBQWSiIiYQp6B1KBBA1xcXHL8XPu8mbi4OBo2bIiHhwdt27bN8TCpP/74g8jISGrWrEnlypXp1asXJ06cKPzWiIjIHSvPQEpMTOTAgQPGz5YtW7BYLMazWJYvX87o0aMZMWIEW7duxc/Pjx49ejg8KGzMmDGsXr2auXPnsnbtWs6dO0fPnj2v+zhnERG59+QZSG5ubnh4eBg/69evp3z58sZzPmJiYujduzfPPvssvr6+TJ48GQ8PD+bNmwfAmTNn+Pjjjxk3bhzt27encePGzJ49mx9++IHNmzff1saJiMid46bmkOx2Ox9//DE9e/akbNmyXL58mb179xIYGOhQLjAw0Hhk9d69e/nzzz8dylStWtV4nLOIiAjc5BNjExMTOXr0KM888wwAmZmZXLlyBavV6lDOarWSnp4OQHp6OiVLlsTV1fW6ZW6Hh7eXg+0Fm6ey9atSSLUREZG83FQgLVy4kKZNm9KgQYPbVR8HBbtbulwxH//eor4qOurroqF+Lnw+Pj43XJ/vQMrIyGDt2rVMmTLFWObq6krJkiXJyMjIUdbd3R0Ad3d3rly5QmZmJm5ubg5lWrRoUaDK31ABz44KfPx7SGpqqvqqiKivi4b6uXjkew5p8eLF3HfffYSHhxvLypQpQ+PGjUlMTHQom5iYiL+/PwCNGzemdOnSDmVOnDjBgQMHjDIiIiL5OkOy2+189NFHdOvWDWdnZ4d1L7zwAgMHDqRZs2b4+/szb948Tp48Sb9+/QCoWLEiffr0YezYsVitVipVqsTrr79O/fr1adeuXaE3SERE7kz5CqRt27Zx6NAh5syZk2Ndt27d+P3335k8eTJpaWnUrVuXpUuX4uXlZZSJioqiZMmS9OvXj0uXLtGmTRtmzZpFyZIlC68lIiJyR7trH2HuMr/gc0i6yi5/NN5edNTXRUP9XDz0XXYiImIKCiQRETEFBZKIiJiCAklERExBgSQiIqagQBIREVNQIImIiCkokERExBQUSCIiYgoKJBERMQUFkoiImIICSURETEGBJCIipqBAEhERU1AgiYiIKSiQRETEFBRIIiJiCgokERExBQWSiIiYggJJRERMQYEkIiKmkK9AOnnyJIMGDaJWrVp4eHjg7+/P9u3bjfV2u52oqCjq1KmDp6cnISEh7N+/32EfNpuNiIgIvLy88PLyIiIiApvNVqiNERGRO1eegWSz2QgKCsJut7N06VKSkpKYNGkSVqvVKDN9+nRiYmKYOHEimzZtwmq1EhYWxrlz54wy/fv3JyUlhfj4eOLj40lJSWHgwIG3p1UiInLHKZVXgRkzZuDp6cns2bONZTVq1DD+3263Exsby7BhwwgNDQUgNjYWHx8f4uPj6devHwcOHGDDhg0kJCTg5+cHwNSpUwkODiY1NRUfH59CbpaIiNxp8jxDWrNmDc2aNaNfv354e3vzyCOPMGfOHOx2OwBHjx4lLS2NwMBAY5uyZcvSsmVLkpKSAEhOTsbZ2Rl/f3+jTEBAAE5OTkYZERG5t+V5hnTkyBHmzp3LkCFDGDZsGPv27WPUqFEAREREkJaWBuAwhJf9799++w2A9PR0XF1dsVgsxnqLxYKbmxvp6enXPXZqaurNt8hQrgDbFsbx7y3qq6Kjvi4a6ufCl9doWJ6BlJWVRZMmTRg7diwAjRo14vDhw8TFxREREVE4tbyOAg3lbT9RvMe/h2jYteior4uG+rl45Dlk5+Hhga+vr8Oy2rVrc/z4cWM9QEZGhkOZjIwM3N3dAXB3dyczM9MY5oOrc0+nTp0yyoiIyL0tz0AKCAjg4MGDDssOHjxItWrVAKhevToeHh4kJiYa6y9dusTOnTuNOSM/Pz/Onz9PcnKyUSY5OZkLFy44zCuJiMi9K89AGjJkCLt372bKlCkcPnyYFStWMGfOHPr37w9cnQsaPHgw06dPZ9WqVfz4448MGTIEJycnunfvDoCvry8dO3Zk+PDhJCcnk5yczPDhwwkKCtJpsYiIAPmYQ2ratCmLFi1i3LhxTJ48mapVq/Laa68ZgQQwdOhQLl68SGRkJDabjWbNmrF8+XLKly9vlImLi2PkyJGEh4cDEBwczKRJk25Dk0RE5E5ksdls9ryL3Xlc5hf8ogZbvyqFUJO7nyaAi476umion4uHvstORERMQYEkIiKmoEASERFTUCCJiIgpKJBERMQUFEgiImIKCiQRETEFBZKIiJiCAklERExBgSQiIqagQBIREVNQIImIiCkokERExBQUSCIiYgoKJBERMQUFkoiImIICSURETEGBJCIipqBAEhERU1AgiYiIKSiQRETEFPIMpKioKFxcXBx+ateubay32+1ERUVRp04dPD09CQkJYf/+/Q77sNlsRERE4OXlhZeXFxEREdhstkJvjIiI3LnydYbk4+PDgQMHjJ8dO3YY66ZPn05MTAwTJ05k06ZNWK1WwsLCOHfunFGmf//+pKSkEB8fT3x8PCkpKQwcOLDwWyMiInesUvkqVKoUHh4eOZbb7XZiY2MZNmwYoaGhAMTGxuLj40N8fDz9+vXjwIEDbNiwgYSEBPz8/ACYOnUqwcHBpKam4uPjU4jNERGRO1W+zpCOHDlCnTp1aNiwIc899xxHjhwB4OjRo6SlpREYGGiULVu2LC1btiQpKQmA5ORknJ2d8ff3N8oEBATg5ORklBEREcnzDKl58+Z88MEH+Pj4cOrUKSZPnkznzp3ZtWsXaWlpAFitVodtrFYrv/32GwDp6em4urpisViM9RaLBTc3N9LT02947NTU1Jtu0P8pV4BtC+P49xb1VdFRXxcN9XPhy2tELM9A6tSpk8O/mzdvTuPGjVm8eDEPP/xwwWqXhwIN520/UbzHv4do6LXoqK+Lhvq5eNz0Zd/Ozs7UqVOHw4cPG/NKGRkZDmUyMjJwd3cHwN3dnczMTOx2u7Hebrdz6tQpo4yIiMhNB9KlS5dITU3Fw8OD6tWr4+HhQWJiosP6nTt3GnNGfn5+nD9/nuTkZKNMcnIyFy5ccJhXEhGRe1ueQ3ZvvPEGjz76KFWrVjXmkP7nf/6HJ598EovFwuDBg4mOjsbHxwdvb2+mTJmCk5MT3bt3B8DX15eOHTsyfPhwpk2bBsDw4cMJCgrSKbGIiBjyDKRff/2V/v37k5mZiZubG82bN2f9+vV4eXkBMHToUC5evEhkZCQ2m41mzZqxfPlyypcvb+wjLi6OkSNHEh4eDkBwcDCTJk26TU0SEZE7kcVms9nzLnbncZlf8IsabP2qFEJN7n6aAC466uuioX4uHvouOxERMQUFkoiImIICSURETEGBJCIipqBAEhERU1AgiYiIKSiQRETEFBRIIiJiCgokERExBQWSiIiYggJJRERMQYEkIiKmoEASERFTUCCJiIgpKJBERMQUFEgiImIKCiQRETEFBZKIiJiCAklERExBgSQiIqZw04EUHR2Ni4sLkZGRxjK73U5UVBR16tTB09OTkJAQ9u/f77CdzWYjIiICLy8vvLy8iIiIwGazFbgBIiJyd7ipQNq9ezcLFiygfv36DsunT59OTEwMEydOZNOmTVitVsLCwjh37pxRpn///qSkpBAfH098fDwpKSkMHDiwcFohIiJ3vHwH0pkzZxgwYADvv/8+Li4uxnK73U5sbCzDhg0jNDSUevXqERsby/nz54mPjwfgwIEDbNiwgWnTpuHn54efnx9Tp05l3bp1pKamFnqjRETkzpPvQMoOnDZt2jgsP3r0KGlpaQQGBhrLypYtS8uWLUlKSgIgOTkZZ2dn/P39jTIBAQE4OTkZZURE5N5WKj+FFi5cyOHDh5kzZ06OdWlpaQBYrVaH5Varld9++w2A9PR0XF1dsVgsxnqLxYKbmxvp6em3XHkREbl75BlIqampjBs3joSEBEqXLl0UdXI49q0rV8zHv7eor4qO+rpoqJ8Ln4+Pzw3X5xlIycnJZGZmEhAQYCy7cuUKO3bsYN68eezatQuAjIwMqlWrZpTJyMjA3d0dAHd3dzIzM7Hb7cZZkt1u59SpU0aZW6n8DW0/cevbFsbx7yGpqanqqyKivi4a6ufikeccUkhICDt27GDbtm3GT5MmTQgPD2fbtm14e3vj4eFBYmKisc2lS5fYuXOnMWfk5+fH+fPnSU5ONsokJydz4cIFh3klERG5d+V5huTi4uJwVR1AuXLlqFSpEvXq1QNg8ODBREdH4+Pjg7e3N1OmTMHJyYnu3bsD4OvrS8eOHRk+fDjTpk0DYPjw4QQFBelTiIiIAPm8qCEvQ4cO5eLFi0RGRmKz2WjWrBnLly+nfPnyRpm4uDhGjhxJeHg4AMHBwUyaNKkwDi8iIncBi81msxd3JW4Hl/kFn0Oy9atSCDW5+2m8veior4uG+rl46LvsRETEFBRIIiJiCgokERExBQWSiIiYggJJRERMQYEkIiKmoEASERFTUCCJiIgpKJBERMQUFEgiImIKCiQRETEFBZKIiJiCAklERExBgSQiIqagQBIREVNQIImIiCkokERExBQUSCIiYgoKJBERMQUFkoiImIICSURETCHPQPrwww9p2bIl1apVo1q1anTq1Il169YZ6+12O1FRUdSpUwdPT09CQkLYv3+/wz5sNhsRERF4eXnh5eVFREQENput0BsjIiJ3rjwDqXLlyrzzzjts2bKFxMRE2rRpw1NPPcX3338PwPTp04mJiWHixIls2rQJq9VKWFgY586dM/bRv39/UlJSiI+PJz4+npSUFAYOHHj7WiUiInecPAMpJCSETp06UbNmTby9vXnzzTdxdnZm9+7d2O12YmNjGTZsGKGhodSrV4/Y2FjOnz9PfHw8AAcOHGDDhg1MmzYNPz8//Pz8mDp1KuvWrSM1NfW2N1BERO4MNzWHdOXKFZYtW8aFCxfw8/Pj6NGjpKWlERgYaJQpW7YsLVu2JCkpCYDk5GScnZ3x9/c3ygQEBODk5GSUERERKZWfQj/88AOdO3fm0qVLODk58cknn1C/fn0jUKxWq0N5q9XKb7/9BkB6ejqurq5YLBZjvcViwc3NjfT09Bset2BnUOUKsG1hHP/eor4qOurroqF+Lnw+Pj43XJ+vQPLx8WHbtm2cPXuWlStXMnjwYL744otCqWBex71l208U7/HvIampqeqrIqK+Lhrq5+KRryG7MmXKULNmTRo3bszYsWNp0KABH3zwAR4eHgBkZGQ4lM/IyMDd3R0Ad3d3MjMzsdvtxnq73c6pU6eMMiIiIrd0H1JWVhaXL1+mevXqeHh4kJiYaKy7dOkSO3fuNOaM/Pz8OH/+PMnJyUaZ5ORkLly44DCvJCIi97Y8h+zefvttOnfuTJUqVYyr57Zv387SpUuxWCwMHjyY6OhofHx88Pb2ZsqUKTg5OdG9e3cAfH196dixI8OHD2fatGkADB8+nKCgIJ0Si4iIIc9ASktLIyIigvT0dCpUqED9+vWJj4+nQ4cOAAwdOpSLFy8SGRmJzWajWbNmLF++nPLlyxv7iIuLY+TIkYSHhwMQHBzMpEmTblOTRETkTmSx2Wz2vIvdeVzmF/yiBlu/KoVQk7ufJoCLjvq6aKifi4e+y05ERExBgSQiIqagQBIREVNQIImIiCkokERExBQUSCIiYgoKJBERMQUFkoiImIICSURETEGBJCIipqBAEhERU1AgiYiIKSiQRETEFBRIIiJiCgokERExBQWSiIiYggJJRERMQYEkIiKmoEASERFTUCCJiIgpKJBERMQU8gyk6Oho2rdvT7Vq1ahVqxY9e/bkxx9/dChjt9uJioqiTp06eHp6EhISwv79+x3K2Gw2IiIi8PLywsvLi4iICGw2W6E2RkRE7lx5BtL27dt5/vnnWbduHatWraJUqVJ07dqV06dPG2WmT59OTEwMEydOZNOmTVitVsLCwjh37pxRpn///qSkpBAfH098fDwpKSkMHDjw9rRKRETuOKXyKrB8+XKHf8+ePRsvLy927dpFcHAwdrud2NhYhg0bRmhoKACxsbH4+PgQHx9Pv379OHDgABs2bCAhIQE/Pz8Apk6dSnBwMKmpqfj4+NyGpomIyJ3kpueQzp8/T1ZWFi4uLgAcPXqUtLQ0AgMDjTJly5alZcuWJCUlAZCcnIyzszP+/v5GmYCAAJycnIwyIiJyb8vzDOnvRo8eTYMGDYwznbS0NACsVqtDOavVym+//QZAeno6rq6uWCwWY73FYsHNzY309PTrHis1NfVmq3eNcgXYtjCOf29RXxUd9XXRUD8XvrxGw24qkF577TV27dpFQkICJUuWLFDF8qNAQ3nbTxTv8e8hGnYtOurroqF+Lh75HrIbM2YMy5YtY9WqVdSoUcNY7uHhAUBGRoZD+YyMDNzd3QFwd3cnMzMTu91urLfb7Zw6dcooIyIi97Z8BdKoUaOMMKpdu7bDuurVq+Ph4UFiYqKx7NKlS+zcudOYM/Lz8+P8+fMkJycbZZKTk7lw4YLDvJKIiNy78hyye/XVV1myZAmffPIJLi4uxpyRk5MTzs7OWCwWBg8eTHR0ND4+Pnh7ezNlyhScnJzo3r07AL6+vnTs2JHhw4czbdo0AIYPH05QUJBOi0VEBMhHIMXFxQEYl3RnGzVqFGPGjAFg6NChXLx4kcjISGw2G82aNWP58uWUL1/eYT8jR44kPDwcgODgYCZNmlRoDRERkTubxWaz2fMududxmV/wixps/aoUQk3ufpoALjrq66Khfi4e+i47ERExBQWSiIiYggJJRERMQYEkIiKmoEASERFTUCCJiIgpKJBERMQUFEgiImIKCiQRETGFm34ektzbcv8GjHI39bgPfQOGiORGZ0giImIKOkO6hxTG9/uJiNwuOkMSERFTUCCJiIgpKJBERMQUFEgiImIKuqhBipwenigiudEZkoiImIICSURETEGBJCIippCvQPr666/p1asXdevWxcXFhUWLFjmst9vtREVFUadOHTw9PQkJCWH//v0OZWw2GxEREXh5eeHl5UVERAQ2m63QGiIiIne2fF3UcOHCBerVq8eTTz7JoEGDcqyfPn06MTExxMTE4OPjw6RJkwgLC2P37t2UL18egP79+3P8+HHi4+MBePnllxk4cCBLliwpxObIvUIXRojcffIVSJ07d6Zz584ADBkyxGGd3W4nNjaWYcOGERoaCkBsbCw+Pj7Ex8fTr18/Dhw4wIYNG0hISMDPzw+AqVOnEhwcTGpqKj4+PoXZJhERuQMVeA7p6NGjpKWlERgYaCwrW7YsLVu2JCkpCYDk5GScnZ3x9/c3ygQEBODk5GSUERGRe1uBAyktLQ0Aq9XqsNxqtZKeng5Aeno6rq6uWCwWY73FYsHNzc0oIyIi9zZT3xibmppagK3LFfPxC9fD2wveHnFkpt/vzbhT632nUT8XvrymZwocSB4eHgBkZGRQrVo1Y3lGRgbu7u4AuLu7k5mZid1uN86S7HY7p06dMsrcSuVv6CYeGHdbjl/YCqE94shUv9980pxr0VA/F48CD9lVr14dDw8PEhMTjWWXLl1i586dxpyRn58f58+fJzk52SiTnJzMhQsXHOaVRETk3pWvM6Tz589z+PBhALKysjh+/DgpKSlUqlSJatWqMXjwYKKjo/Hx8cHb25spU6bg5ORE9+7dAfD19aVjx44MHz6cadOmATB8+HCCgoL0KUSKjS4dFzGXfAXSt99+y+OPP278OyoqiqioKJ588kliY2MZOnQoFy9eJDIyEpvNRrNmzVi+fLlxDxJAXFwcI0eOJDw8HIDg4GAmTZpUyM0REZE7lcVms9mLuxK3w9326VePHzenon6NaG6jaKifi4e+y05ERExBgSQiIqagQBIREVMw9Y2xdwPN/YiI5I/OkERExBQUSCIiYgoashMpgLvt9gKR4qQzJBERMQUFkoiImIICSURETEGBJCIipqBAEhERU1AgiYiIKSiQRETEFHQfkkgxu7l7mcrl+jh73cskdwMFkshdQDfoyt1AQ3YiImIKCiQRETEFBZKIiJiC5pBEBNA8lBQ/BdIN6OF6IjdHoSYFUeRDdnFxcTRs2BAPDw/atm3Ljh07iroKIiJiQkUaSMuXL2f06NGMGDGCrVu34ufnR48ePTh27FhRVkNEREzIYrPZ7EV1sA4dOlC/fn1mzJhhLGvatCmhoaGMHTu2UI+l4TaRe1tBhv5SU1Px8fHREGQRK7I5pMuXL7N3715eeuklh+WBgYEkJSUV+vH0IhCRW+Xj4wPofaSoFdmQXWZmJleuXMFqtTost1qtpKenF1U1RETEpHQfkoiImEKRBZKrqyslS5YkIyPDYXlGRgbu7u5FVQ0RETGpIgukMmXK0LhxYxITEx2WJyYm4u/vX1TVEBERkyrSG2NfeOEFBg4cSLNmzfD392fevHmcPHmSfv36FWU1RETEhIp0Dqlbt25ERUUxefJkWrduza5du1i6dCleXl6FehzdfFsw0dHRtG/fnmrVqlGrVi169uzJjz/+6FDGbrcTFRVFnTp18PT0JCQkhP379zuUsdlsRERE4OXlhZeXFxEREdhstiJsyZ0lOjoaFxcXIiMjjWXq58Jz8uRJBg0aRK1atfDw8MDf35/t27cb69XXxa/IL2ro378/+/btIz09nS1bttCqVatC3b9uvi247du38/zzz7Nu3TpWrVpFqVKl6Nq1K6dPnzbKTJ8+nZiYGCZOnMimTZuwWq2EhYVx7tw5o0z//v1JSUkhPj6e+Ph4UlJSGDhwYHE0yfR2797NggULqF+/vsNy9XPhsNlsBAUFYbfbWbp0KUlJSUyaNMnhql/1dfEr0htji0JR3nx7rzh//jxeXl4sWrSI4OBg7HY7derUYcCAAbz66qsAXLx4ER8fH95991369evHgQMH8Pf3JyEhgYCAAAB27txJcHAwu3fvNu7zEDhz5gxt27ZlxowZTJw4kXr16jF58mT1cyEaN24cX3/9NevWrct1vfraHO6qy76zb74NDAx0WH67br69V5w/f56srCxcXFwAOHr0KGlpaQ79XLZsWVq2bGn0c3JyMs7Ozg4XrAQEBODk5KTfxd8MGzaM0NBQ2rRp47Bc/Vx41qxZQ7NmzejXrx/e3t488sgjzJkzB7v96udx9bU53FWBpJtvb4/Ro0fToEED/Pz8AEhLSwO4YT+np6fj6uqKxWIx1lssFtzc3PS7uMbChQs5fPgwb7zxRo516ufCc+TIEebOnUuNGjVYtmwZgwYN4p133uHDDz8E1NdmocdPyA299tpr7Nq1i4SEBEqWLFnc1bmrpKamMm7cOBISEihdunRxV+eulpWVRZMmTYxh+0aNGnH48GHi4uKIiIgo5tpJtrvqDEk33xauMWPGsGzZMlatWkWNGjWM5R4eHgA37Gd3d3cyMzONIRG4Ok5/6tQp/S7+V3JyMpmZmQQEBODq6oqrqytff/01cXFxuLq6cv/99wPq58Lg4eGBr6+vw7LatWtz/PhxYz2or4vbXRVIuvm28IwaNcoIo9q1azusq169Oh4eHg79fOnSJXbu3Gn0s5+fH+fPnyc5Odkok5yczIULF/S7+F8hISHs2LGDbdu2GT9NmjQhPDycbdu24e3trX4uJAEBARw8eNBh2cGDB6lWrRqg17RZ3HVDdrr5tuBeffVVlixZwieffIKLi4sxvu7k5ISzszMWi4XBgwcTHR2Nj48P3t7eTJkyBScnJ7p37w6Ar68vHTt2ZPjw4UybNg2A4cOHExQUpKuR/peLi4txoUi2cuXKUalSJerVqwegfi4kQ4YMoXPnzkyZMoVu3bqRkpLCnDlzePPNNwH0mjaJu+6yb7h6Y+z06dNJS0ujbt26/Otf/yr0+53uZn9/k8w2atQoxowZA1wdqpgwYQILFizAZrPRrFkzpkyZYryRwtV7P0aOHMmXX34JQHBwMJMmTbru/uXqWVP2Zd+gfi5M69atY9y4cRw8eJCqVasyYMAABg4caFykoL4ufndlIImIyJ3nrppDEhGRO5cCSURETEGBJCIipqBAEhERU1AgiYiIKSiQRETEFBRIImIaDRo0YPDgwcVdDSkmCiQpNqdOneLtt98mICCAypUr88ADD9CyZUvefvttfvvtt9tyzKSkJKKiou7ap3za7XY+//xzOnfuTM2aNalcuTKNGzemX79+bNiwobirJ3JDd91XB8mdYe/evfTo0YOzZ88SHh7OgAEDKFGiBD/88AMfffQRq1evZs+ePYV+3OTkZCZOnEjv3r3vyrvrR40axZw5cwgKCuLVV1/lH//4B4cPHyYxMZFly5bRsWPH4q6iyHUpkKTInTlzhqeffhqAzZs3U7duXYf1b775JtOnTy+Oqpne5cuXKVGiBKVK5fzTTU9PJy4ujieffJLY2Nhc14uYmYbspMgtWLCA48eP89577+UII4CKFSvy1ltvGf++3rxCSEgIISEhDsvi4uJo2bIllStXxsvLi0ceeYT58+cDEBUVZXyZZqNGjYwvN922bZux/bx582jRogUeHh7Url2bYcOGcfr06RzHffjhh/nhhx947LHHeOCBB2jUqBHLly8Hrj7WumPHjnh6etK8eXM2btyYo+4nT57kpZdeonbt2ri7u+Pn58fcuXMdymzbtg0XFxeWLl1KVFQUDz30EJ6enpw4cSLXfj169ChZWVm0aNEi1/XXPiLh2n3/61//ok6dOjzwwAN069aNQ4cO5dj24MGD9O3blwcffBAPDw9at27NypUrHcosWrQIFxcXtm/fzmuvvUatWrWoXLkyTz31FKdOnXIoa7fbmTx5MvXq1eOBBx7gn//8J/v378+13nLv0BmSFLkvv/ySf/zjH4SFhRXqfj/66CNeffVVQkNDGTBgAH/++Sc//fQTSUlJ9OvXj8cff5xDhw4RHx/Pv/71L1xdXQGM5+RMnjyZ8ePH06ZNG/r27cvPP/9MXFwc33zzDRs3buS+++4zjnX27Fl69uxJ165d6dq1K/Pnz2fAgAHY7XbGjBlDv379CA8P5/3336dv3758//33VKxYEbj6jJ2OHTuSlZXF888/j9VqZcuWLYwYMYLff/+dyMhIh3ZFR0dTokQJBg4cCICzs3Ou7c9+lMKqVasIDw/Hyckpzz6bNm0aWVlZvPjii9hsNmbPns3jjz/O119/TaVKlQA4cOAAnTt3xsPDg6FDh+Lk5MQXX3zBs88+y+zZs+nZs6fDPseMGUOlSpUYNWoUv/zyC7GxsURGRhofDADGjx/PlClT6Ny5M506dWLfvn2Eh4fzxx9/5FlnuXspkKTI/fTTT3h7e1OmTJlC3e+6deuoW7cuCxcuzHX9Qw89RKNGjYiPjyckJITq1asb606dOsXkyZNp27Yty5cvN56O26BBA1544QUWLlzo8GTRtLQ0Zs2aRa9evQBo3749Dz/8MP379+fLL78kICAAuBp23bp1Y+XKlTzzzDMAvPfee1y+fJkdO3bg5uYGwHPPPcfLL79MdHQ0AwYMcJjfOn/+PElJSXkGjKenJ71792bx4sXUrVuXVq1aERAQQIcOHXjooYdy3SYjI4Pdu3cbx2vdujWhoaHExMQYj1UfPXo0DzzwAImJiZQtWxaAAQMGEBYWxjvvvMMTTzzh8Fjv+++/nxUrVhjLsrKymD17NmfOnKFixYqcOnWKGTNmEBQUxGeffWaUGz9+vPEt53Jv0pCdFLlz585Rvnz5Qt9vhQoVOHHiBP/9739vetvNmzdz+fJlBg0a5PCo9l69euHu7s5XX33lUL5s2bI88cQTxr99fHyoWLEi3t7eRhgBNGvWDIAjR44AV4eqVq5cSefOnbFYLGRmZho/gYGBXLx4McfFHL169crX2Q7AjBkzmDBhAl5eXiQkJDB27FgeeeQRAgMDSU1NzVG+V69eDuHXtm1b6tatS0JCAgCnT59m8+bNdO3alf/5n/9xqG+HDh349ddfczz4rk+fPg4B1aJFC65cucKxY8eA/+vr559/3qHcoEGD8tVGuXvpDEmKXPny5Tl37lyh73fYsGFs3bqVwMBAatSoQfv27QkLC6NNmzZ5bpv9Zvn3B62VLFmSWrVq8csvvzgsf+CBByhRwvHzXIUKFahSpYrDsuxhuuzLzE+dOoXNZuOTTz7hk08+ybUuf3+M9rWPj89LqVKlGDRoEIMGDcJms/HNN9+wePFili9fTq9evdixY4fD0GOtWrVy7KNWrVps3boVgMOHDxvPCZowYcJ163ttv1WtWtVhfXbgZfdBdl///diurq535ZWPkn8KJClyvr6+fPfdd1y+fDlfw3bXfoq+VlZWlkMo+Pr6snv3btavX8/GjRtZt24d8+fPp3///kyZMqXQ6g84nEXlZ7ndbjfqDNC9e3fjSsO/q1OnjsO/s4fJbpaLiwsdO3akY8eOlC5dmiVLlvDNN9/c1MMqs+ub/cTV3Fz7ADvIuw9ErkeBJEUuODiYpKQkVqxY4TDsdT0uLi6cOXMmx/Jjx445zAPB1UeAh4aGEhoayl9//cWQIUOIi4vjlVdeoXLlytc9RvYFAampqXh7exvLs7KyOHToEA0bNsxv827Izc2N8uXL89dff9GuXbtC2Wd+NGvWjCVLlnDy5EmH5bldUXfo0CG8vLyA/zs7K1WqVKHVN7uvDx065HCWlJmZedfesCz5ozkkKXJ9+/alcuXKvPHGGxw4cCDH+nPnzvHuu+8a/37wwQf55ptvuHz5srEsISGB48ePO2z3+++/O/y7VKlSxqf37EDLnov5+xtf+/btKVOmDLNnzzbOCgCWLl1Keno6QUFBt9DSnEqWLEmXLl1Ys2YN+/bty7H+75dH34y0tDR+/PHHXNdlf0vD34ckP/vsM4e+2LJlC/v37zfaa7Vaad26NQsXLuTXX38tlPq2a9eO0qVLM3fuXIezplmzZt30vuTuojMkKXIuLi4sWrSIHj160LZtW7p3707Tpk0pUaIEP/74I8uWLaNSpUrGPUPPPPMMK1euJDw8nLCwMH7++WeWLl3Kgw8+6LDfsLAwrFYrAQEBuLu78/PPPzNnzhzq169vXNrdpEkTAMaNG0f37t0pU6YMbdq0wWq1EhkZyfjx4wkLCyMkJIQjR47w4Ycf8tBDDxlXyBWGt99+m+3bt9O5c2eeeeYZ6tati81mY9++fXzxxRekpaXd0n5//fVXAgMDeeSRR2jbti2enp6cPn2atWvXsnPnTrp06ZLjTM9qtfLoo4/y9NNPc+bMGWbNmoWnpycvvPCCUSY6OpqgoCBatWrFs88+y4MPPkhGRgbffPMNBw4c4Ntvv72perq5ufHSSy8RHR1Nz5496dSpE99//z3r1683LsWXe5MCSYpFkyZN2LlzJ++//z4JCQksW7YMu91OzZo16du3r3HPDUCHDh147733+OCDDxgzZgxNmjRhyZIlvP766w777NevH59//jmxsbGcO3cOT09PnnrqKSIjI425piZNmjB27Fjmzp3LCy+8QFZWFqtXrzYCydXVlTlz5vDGG29QsWJFevfuzdixYx0uBCgoq9XKxo0bmTRpEmvWrGHevHlUqlSJ2rVr8957793yfr29vZkwYQLr169nwYIFpKenU6ZMGby9vXnvvfcc+jTbsGHDSE1NZebMmZw5c4YWLVowadIk7r//fqOMj48PiYmJTJw4kc8++4zMzEzc3Nx46KGHcvwO8uuNN97gvvvuY/78+Wzbto1mzZqxbNmyfA3hyt3LYrPZNNMoco/Ztm0bjz/+OHPnziU8PLy4qyMCaA5JRERMQoEkIiKmoEASERFT0BySiIiYgs6QRETEFBRIIiJiCgokERExBQWSiIiYggJJRERMQYEkIiKm8P8BJoeZdA3HnMYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(np.concatenate([\n",
    "    np.random.gamma(5, 50, 1000), \n",
    "    np.zeros(700)\n",
    "]), bins=20)\n",
    "plt.xlabel(\"Customer Spend\")\n",
    "plt.title(\"Distribution Customer Spend\");"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터 과학자가 이것을 보았을 때, 머릿속에 떠오르는 첫 번째 생각은 모델링을 두 단계로 나누는 것입니다. 첫 번째는 참여, 즉 $Y>0$일 확률입니다. 이 지출 예에서 이것은 고객의 지출 여부를 결정하는 모형입니다. 두 번째는 참여하기로 결정한 사람들을 위해 Y를 모델링합니다. 이것은 Conditional-on-Positives 효과입니다. 이 경우, 고객이 무엇이든 지출하기로 결정한 후 지출금액입니다. 지출에 대한 T의 효과를 추정하고자 한다면 다음과 같이 정리할 수 있습니다.\n",
    " \n",
    "$\n",
    "E[Y|T] = E[Y|Y>0, T]P(Y>0|T)\n",
    "$\n",
    " \n",
    "참여 모델 $P(Y_i>0|T_i)$에는 아무 문제가 없습니다. $T$가 랜덤하게 할당되면 treatment로 인한 지출 확률 증가를 포착할 수 있습니다. 또한 위의 분해에는 문제가 없습니다. 전체 확률의 법칙(law of total probability)에 따라 수학적으로 맞는 말입니다.\n",
    " \n",
    "문제는 COP part를 추정하는 것에 있습니다. **무작위로 할당되더라도 편향될 수 있습니다**. 직관적인 수준에서 일부 단위는 treatment를 받지 않았기 때문에 $0$이라고 생각하는 것은 미친 것이 아닙니다. treatment로 인해 $0$이 되지 않습니다. 반면에, 일부 유닛은 절대로 $0$이 아닙니다. treatment은 outcome을 높일 수 있지만, treatment가 없더라도 그것들은 $0$이 되지 않습니다. 여기서 중요한 것은 이 두 가지 유형의 유닛은 비교할 수 없다는 것을 이해하는 것입니다. 0이 아닌 사람들은 treatment를 받지 않으면 0인 사람들에 비해 $Y_0$이 높습니다. 사실, 후자의 경우 $Y_0 = 0$입니다.\n",
    " \n",
    "이 점에 유의하여 0을 제거하면 treatment그룹과 통제그룹 양쪽에 0이 유지됩니다. 하지만 우리는 treatment를 통해 0에서 0이 아닌 것으로 바뀌는 것들을 통제그룹에서 제거할 것입니다. 이것은 treatment와 대조군을 더 이상 비교할 수 없게 만들 것입니다. 왜냐하면 통제그룹에는 0이 없고 $Y_0$에 높은 값만 포함되는 반면 treatment를 받은 값에는 두 가지 유형의 단위가 모두 포함되기 때문입니다.\n",
    " \n",
    "이제 문제를 직관적으로 이해했으므로 수학적 각도에서 확인해보겠습니다. 이것을 보기 위해 treatment 효과를 나눠보겠습니다. 랜덤 할당에서는 평균의 차이와 같습니다.\n",
    " \n",
    "$$\n",
    "\\begin{align*} \n",
    "&E[Y|T=1] - E[Y|T=0]\\\\\n",
    "&=E[Y|Y>0, T=1]P(Y>0|T=1) - E[Y|Y>0, T=0]P(Y>0|T=0)\\\\\n",
    "&=\\underbrace{\\{P(Y>0|T=1) - P(Y>0|T=0)\\}}_{Participation \\ Effect} * E[Y|Y>0, T=1]\\\\\n",
    "&+\\underbrace{\\{E[Y|Y>0, T=1] - E[Y|Y>0, T=0]\\}}_{COP \\ Effect} * P(Y>0|T=0)\n",
    "\\end{align*} \n",
    "$$\n",
    " \n",
    "마지막 식은 $E[Y_i|Y_i>0, T_i=1]P(Y_i>0|T_i=0)$ 를 더하고 빼서 재배열 하는 것입니다. 이것은 평균의 차이가 두 부분으로 구성된다는 것을 의미합니다. 첫 째, 결과 $y$가 양수일 확률의 차이입니다. 이것은 고객이 지출에 참여할 확률의 증가를 측정하기 때문에 참여 효과라고 불립니다. 둘째, 참여에 따라 결정되는 outcome의 차이인 COP 효과입니다. 지금까지는 좋습니다. 이 식에는 아무 문제가 없습니다. 이것은 수학적 진리입니다. 문제는 우리가 각 부분을 개별적으로 추정하려고 할 때 발생합니다. \n",
    "\n",
    "이것은 COP 효과를 분석하면 더욱 분명해집니다.\n",
    " \n",
    "$$\n",
    "\\begin{align*} \n",
    "E[Y|Y>0, T=1] - E[Y|Y>0, T=0]&=E[Y_{1}|Y_{1}>0]-E[Y_{0}|Y_{0}>0] \\\\\n",
    "&=\\underbrace{E[Y_{1} - Y_{0}|Y_{1}>0]}_{Causal \\ Effect} + \\underbrace{\\{ E[Y_{0}|Y_{1}>0] - E[Y_{0}|Y_{0}>0] \\}}_{Selection \\ Bias}\n",
    "\\end{align*} \n",
    "$$\n",
    " \n",
    "$E[Y_{i0}|Y_{i1}>0]$를 더하고 뺀 후에 두 번째 식이 도출됩니다. COP 효과를 분해할 때 먼저 참가자 하위 모집단에 대한 인과효과를 얻습니다. 이 예시에서, 이것은 무언가를 소비하기로 결정한 사람들의 인과효과가 될 것입니다. 둘 째, treatment에 할당되었을 때 참여하기로 결정한 사람들($E[Y_{i0}|Y_{i1}>0]$)과 treatment 없이도 참여한 사람들($E[Y_{i0}|Y_{i0}>0]$)에 대한 $Y_0$의 차이인 편향 항을 얻습니다. 우리의 경우,이 편향은 아마도 부정적일 것입니다. 왜냐하면 treatment에 할당되었을 때 treatment를 받지 않은 사람들은 treatment $E[Y_{i0}|Y_{i1}>0] < E[Y_{i0}|Y_{i0}>0]$ 가 없어도 지출하는 사람들보다 적을 것이기 때문입니다.\n",
    " \n",
    "![img](./data/img/beyond-conf/cop.png)\n",
    " \n",
    "자, COP 편향이 처음에는 매우 직관적이지 않다는 것을 알고 있기에, 시각적인 예를 통해서 한번 살펴보면 좋을 것 같습니다. 마케팅 캠페인을 통해 사람들이 우리 제품에 얼마나 많은 비용을 지출하는지 추정하려고 해요. 이번 마케팅 캠페인은 무작위로 선정되었으므로 교란요인을 걱정할 필요가 없습니다. 이 예에서는, 고객을 2개의 세그먼트(segment)로 나눌 수 있습니다. 첫째, 마케팅 캠페인을 봐야만 우리 제품을 살 수 있는 사람들이 있습니다. 이 고객들을 알뜰 고객이라고 부르겠습니다. 그들은 우리가 그들을 계속 푸쉬하지 않으면 지출을 하지 않습니다. 그리고 캠페인이 없어도 소비하는 고객이 있습니다. 캠페인은 그들이 더 많은 돈을 쓰게 만들지만, 어쨌든 그들은 이미 캠페인에 참여하지 않고 소비를 할 것입니다. 그들을 부자 고객이라고 부르겠습니다. 그림에서 나는 반사실사건(counterfactuals)를 밝은 색상과 파선으로 표시했습니다.\n",
    "\n",
    " \n",
    "![img](./data/img/beyond-conf/cop-ex1.png)\n",
    " \n",
    "캠페인에 대한 ATE를 추정하기 위해, 우리는 무작위화를 진행했기 때문에 treatment를 받은 것과 treatment를 받지 않은 것에 대해 비교하기만 하면 됩니다. 그러나 추정을 두 가지 모델, 즉 $P(Y_i>0|T_i)$를 추정하는 참여 모델과 $E[Y_i|Y_i>0]$ 을 추정하는 COP로 나누는 COP 공식을 사용한다고 가정합니다. 이렇게 하면 지출하지 않은 모든 사람이 분석에서 제거됩니다.\n",
    " \n",
    "![img](./data/img/beyond-conf/cop-ex2.png)\n",
    " \n",
    "이렇게 할 경우 treatment 그룹과 통제 그룹은 더 이상 비교할 수 없습니다. 보시다시피, treatment를 받지 않은 사람들은 이제 캠페인 없이도 소비할 고객들의 일부로만 구성되어 있습니다. 또한 우리는 여기서 편향의 방향을 알 수 있다는 것을 알아두세요. $E[Y_{i0}|Y_{i1}>0] - E[Y_{i0}|Y_{i0}>0]$ 혹은 $E[Y_{i0}|\\text{Frugal and Rich}] - E[Y_{i0}|Rich]$ 일 것입니다. 이것은 부자들이 검소한 고객들보다 더 많은 돈을 쓰기 때문에 분명히 부정적입니다. 결과적으로 참가자 모집단만 필터링하면 무작위화로 인해 처음에는 편향이 없더라도 ATE에 대한 추정치가 편향됩니다. 저는 이것이 당신이 전염병처럼 COP를 피하도록 설득하기를 진심으로 바랍니다. 저는 너무 많은 데이터 과학자들이 이것이 가져오는 문제들을 알지 못한 채 이 별도의 추정을 하고 있는 것을 봅니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "선택 편향을 마무리하기 위해, 우리는 항상 treatment와 outcome 사이에 있거나 outcome과 treatment의 공통적인 효과인 변수를 통제하지 말아야 한다는 것을 스스로에게 상기시킬 필요가 있습니다. 그래프를 통해 잘못된 통제 예시를 보면 다음과 같습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n -->\n<!-- Pages: 1 -->\n<svg width=\"89pt\" height=\"260pt\"\n viewBox=\"0.00 0.00 89.00 260.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 256)\">\n<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-256 85,-256 85,4 -4,4\"/>\n<!-- T -->\n<g id=\"node1\" class=\"node\">\n<title>T</title>\n<ellipse fill=\"none\" stroke=\"gold\" cx=\"54\" cy=\"-234\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"54\" y=\"-230.3\" font-family=\"Times,serif\" font-size=\"14.00\">T</text>\n</g>\n<!-- X_1 -->\n<g id=\"node2\" class=\"node\">\n<title>X_1</title>\n<ellipse fill=\"none\" stroke=\"red\" cx=\"27\" cy=\"-162\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"27\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\">X_1</text>\n</g>\n<!-- T&#45;&gt;X_1 -->\n<g id=\"edge1\" class=\"edge\">\n<title>T&#45;&gt;X_1</title>\n<path fill=\"none\" stroke=\"black\" d=\"M47.6,-216.41C44.49,-208.34 40.67,-198.43 37.17,-189.35\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"40.4,-188.03 33.54,-179.96 33.87,-190.55 40.4,-188.03\"/>\n</g>\n<!-- X_2 -->\n<g id=\"node4\" class=\"node\">\n<title>X_2</title>\n<ellipse fill=\"none\" stroke=\"red\" cx=\"54\" cy=\"-18\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"54\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">X_2</text>\n</g>\n<!-- T&#45;&gt;X_2 -->\n<g id=\"edge3\" class=\"edge\">\n<title>T&#45;&gt;X_2</title>\n<path fill=\"none\" stroke=\"black\" d=\"M57.65,-215.91C59.68,-205.57 61.98,-192.09 63,-180 67.03,-132.17 67.03,-119.83 63,-72 62.28,-63.5 60.93,-54.31 59.49,-46.01\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"62.91,-45.29 57.65,-36.09 56.03,-46.56 62.91,-45.29\"/>\n</g>\n<!-- Y -->\n<g id=\"node3\" class=\"node\">\n<title>Y</title>\n<ellipse fill=\"none\" stroke=\"black\" cx=\"27\" cy=\"-90\" rx=\"27\" ry=\"18\"/>\n<text text-anchor=\"middle\" x=\"27\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">Y</text>\n</g>\n<!-- X_1&#45;&gt;Y -->\n<g id=\"edge2\" class=\"edge\">\n<title>X_1&#45;&gt;Y</title>\n<path fill=\"none\" stroke=\"black\" d=\"M27,-143.7C27,-135.98 27,-126.71 27,-118.11\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"30.5,-118.1 27,-108.1 23.5,-118.1 30.5,-118.1\"/>\n</g>\n<!-- Y&#45;&gt;X_2 -->\n<g id=\"edge4\" class=\"edge\">\n<title>Y&#45;&gt;X_2</title>\n<path fill=\"none\" stroke=\"black\" d=\"M33.4,-72.41C36.51,-64.34 40.33,-54.43 43.83,-45.35\"/>\n<polygon fill=\"black\" stroke=\"black\" points=\"47.13,-46.55 47.46,-35.96 40.6,-44.03 47.13,-46.55\"/>\n</g>\n</g>\n</svg>\n",
      "text/plain": [
       "<graphviz.dot.Digraph at 0x12ce90b38>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g = gr.Digraph()\n",
    "\n",
    "g.edge(\"T\", \"X_1\"), g.node(\"T\", color=\"gold\"), g.edge(\"X_1\", \"Y\"), g.node(\"X_1\", color=\"red\")\n",
    "g.edge(\"T\", \"X_2\"), g.edge(\"Y\", \"X_2\"), g.node(\"X_2\", color=\"red\")\n",
    "\n",
    "g"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key Ideas\n",
    "\n",
    "해당 챕터에서 인과관계를 확인하기 위해 우리의 모델에 변수를 추가해야 하는지, 그렇지 않은지를 살펴보았습니다. outcome에 대해 좋은 예측변수는 $T$(교란요인이 아닌)를 예측하지 않더라도 모형에 추가해야 합니다. 이는 $Y$를 예측하면 분산이 낮아지고 인과 관계를 추정할 때 통계적으로 유의미한 결과가 나타날 가능성이 높아지기 때문입니다. 다음으로, 우리는 treatment를 예측하지만 outcome을 예측하지 않는 변수를 추가하는 것이 좋지 않다는 것을 알았습니다. 이러한 변수들은 treatment의 변동성을 줄여 인과효과를 찾기 어렵게 만듭니다. 마지막으로 선택 편향에 대해 살펴보았습니다. 이것은 treatment에서 outcome 또는 treatment와 outcome의 공통적인 효과인 변수에 대한 인과 경로의 변수를 통제할 때 발생하는 편향입니다.\n",
    "\n",
    "## References\n",
    "\n",
    "이 책을 쓰기 위해 Joshua Angrist, Alberto Abadie, Christopher Walters의 대단한 계량 경제학 수업 자료를 많이 참고했습니다. 해당 자료에 있는 대부분의 아이디어는 전미경제학회(American Economic Association)의 수업에서 가져왔어요. 이렇게 좋은 참고자료를 보는 것이 2020년의 힘든 한 해를 지탱하게 만들어준 원동력이었다고 생각해요.\n",
    "\n",
    "* [Cross-Section Econometrics](https://www.aeaweb.org/conference/cont-ed/2017-webcasts)\n",
    "* [Mastering Mostly Harmless Econometrics](https://www.aeaweb.org/conference/cont-ed/2020-webcasts)\n",
    "\n",
    "또한, Angrist의 정말 좋은 책들을 참고자료에 넣고 싶어요. 해당 저자가 쓴 책들은 계량경제학(Econometrics) 또는 '메트릭스(Metrics, 계량적 분석)'가 매우 유용할 뿐만 아니라 매우 흥미롭다는 걸 알려주었어요.\n",
    "\n",
    "* [Mostly Harmless Econometrics](https://www.mostlyharmlesseconometrics.com/)\n",
    "* [Mastering ‘Metrics](https://www.masteringmetrics.com/)\n",
    "\n",
    "마지막으로 참고한 자료는 Miguel Hernan과 Jamie Robins의 책입니다. 이 책들은 제가 대답해야 했던 까다로운 인과적인 질문에서 신뢰할 수 있는 동반자 같은 존재였어요.\n",
    "\n",
    "* [Causal Inference Book](https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/)\n",
    "\n",
    "![img](https://matheusfacure.github.io/python-causality-handbook/_images/poetry.png)\n",
    "\n",
    "해당 챕터에 사용된 데이터는 Susan Athey과 Stefan Wager [Estimating Treatment Effects with Causal Forests: An Application](https://arxiv.org/pdf/1902.07409.pdf)에서 제공하고 있습니다.\n",
    "\n",
    "## Contribute\n",
    "\n",
    "Causal Inference for the Brave and True는 인과추론, 통계학에 대한 오픈소스 자료입니다. 이 자료는 금전적으로나 모든 분들이 쉽게 접근하실 수 있도록 하는 것이 목표입니다. 또한, 이 책은 Python 기반의 무료 소프트웨어만 사용해요.\n",
    "여러분들께서 이 자료가 가치 있다고 생각하시고, 금전적으로 지원을 원하신다면 [Patreon](https://www.patreon.com/causal_inference_for_the_brave_and_true)를 방문해주세요. \n",
    "만약 여러분이 금전적으로 기여하기가 쉽지 않으시다면, 오타 수정, 수정 제안, 이해하기 난해한 부분에 대한 피드백 제공 등을 통해 도움을 주실 수 있어요. 이 책의 Github 저장소 [이슈 페이지](https://github.com/CausalInferenceLab/Causal-Inference-with-Python/issues)를 방문해주세요. 마지막으로 이 자료가 여러분의 마음에 드셨다면 도움이 될 수 있는 다른 사람들과 공유해주시고, [한국어 번역 자료](https://github.com/CausalInferenceLab/Causal-Inference-with-Python/stargazers)와 [해당 번역본의 원서](https://github.com/matheusfacure/python-causality-handbook/stargazers)에 star 부탁드립니다!"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Workspace_JS",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "vscode": {
   "interpreter": {
    "hash": "a812027cb36d52afc6bf34b1f842ed1960d479f5dea7272b4a52eb53764f32d8"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}