Linear+Regression+-+Predicting+House+Prices.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Linear Regression - Predicting House Prices \n",
    "\n",
    "Housing data for the city of Ames, Iowa, United States, from the years 2006 to 2010, was used to train a Linear Regression model to predict the sale price of a house, based on its features.\n",
    "\n",
    "We'll start by importing some of the libraries we'll need, as well as briefly exploring the data, and then setting up a pipeline of functions that will allow us to quickly iterate on different models.\n",
    "\n",
    "The dataset can be found [here](https://dsserver-prod-resources-1.s3.amazonaws.com/235/AmesHousing.txt). This dataset was originally compiled by Dean De Cock for the primary purpose of having a high quality dataset for regression.\n",
    "\n",
    "Information on the columns in the dataframe can be found [here](http://jse.amstat.org/v19n3/decock/DataDocumentation.txt).\n",
    "\n",
    "## Introduction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Order</th>\n",
       "      <th>PID</th>\n",
       "      <th>MS SubClass</th>\n",
       "      <th>MS Zoning</th>\n",
       "      <th>Lot Frontage</th>\n",
       "      <th>Lot Area</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>Lot Shape</th>\n",
       "      <th>Land Contour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>Lot Config</th>\n",
       "      <th>Land Slope</th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Condition 1</th>\n",
       "      <th>Condition 2</th>\n",
       "      <th>Bldg Type</th>\n",
       "      <th>House Style</th>\n",
       "      <th>Overall Qual</th>\n",
       "      <th>Overall Cond</th>\n",
       "      <th>Year Built</th>\n",
       "      <th>Year Remod/Add</th>\n",
       "      <th>Roof Style</th>\n",
       "      <th>Roof Matl</th>\n",
       "      <th>Exterior 1st</th>\n",
       "      <th>Exterior 2nd</th>\n",
       "      <th>Mas Vnr Type</th>\n",
       "      <th>Mas Vnr Area</th>\n",
       "      <th>Exter Qual</th>\n",
       "      <th>Exter Cond</th>\n",
       "      <th>Foundation</th>\n",
       "      <th>Bsmt Qual</th>\n",
       "      <th>Bsmt Cond</th>\n",
       "      <th>Bsmt Exposure</th>\n",
       "      <th>BsmtFin Type 1</th>\n",
       "      <th>BsmtFin SF 1</th>\n",
       "      <th>BsmtFin Type 2</th>\n",
       "      <th>BsmtFin SF 2</th>\n",
       "      <th>Bsmt Unf SF</th>\n",
       "      <th>Total Bsmt SF</th>\n",
       "      <th>Heating</th>\n",
       "      <th>Heating QC</th>\n",
       "      <th>Central Air</th>\n",
       "      <th>Electrical</th>\n",
       "      <th>1st Flr SF</th>\n",
       "      <th>2nd Flr SF</th>\n",
       "      <th>Low Qual Fin SF</th>\n",
       "      <th>Gr Liv Area</th>\n",
       "      <th>Bsmt Full Bath</th>\n",
       "      <th>Bsmt Half Bath</th>\n",
       "      <th>Full Bath</th>\n",
       "      <th>Half Bath</th>\n",
       "      <th>Bedroom AbvGr</th>\n",
       "      <th>Kitchen AbvGr</th>\n",
       "      <th>Kitchen Qual</th>\n",
       "      <th>TotRms AbvGrd</th>\n",
       "      <th>Functional</th>\n",
       "      <th>Fireplaces</th>\n",
       "      <th>Fireplace Qu</th>\n",
       "      <th>Garage Type</th>\n",
       "      <th>Garage Yr Blt</th>\n",
       "      <th>Garage Finish</th>\n",
       "      <th>Garage Cars</th>\n",
       "      <th>Garage Area</th>\n",
       "      <th>Garage Qual</th>\n",
       "      <th>Garage Cond</th>\n",
       "      <th>Paved Drive</th>\n",
       "      <th>Wood Deck SF</th>\n",
       "      <th>Open Porch SF</th>\n",
       "      <th>Enclosed Porch</th>\n",
       "      <th>3Ssn Porch</th>\n",
       "      <th>Screen Porch</th>\n",
       "      <th>Pool Area</th>\n",
       "      <th>Pool QC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>Misc Feature</th>\n",
       "      <th>Misc Val</th>\n",
       "      <th>Mo Sold</th>\n",
       "      <th>Yr Sold</th>\n",
       "      <th>Sale Type</th>\n",
       "      <th>Sale Condition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>526301100</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>141.0</td>\n",
       "      <td>31770</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>1960</td>\n",
       "      <td>1960</td>\n",
       "      <td>Hip</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>Plywood</td>\n",
       "      <td>Stone</td>\n",
       "      <td>112.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>Gd</td>\n",
       "      <td>Gd</td>\n",
       "      <td>BLQ</td>\n",
       "      <td>639.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>441.0</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>Fa</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>1656</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1656</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>7</td>\n",
       "      <td>Typ</td>\n",
       "      <td>2</td>\n",
       "      <td>Gd</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1960.0</td>\n",
       "      <td>Fin</td>\n",
       "      <td>2.0</td>\n",
       "      <td>528.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>P</td>\n",
       "      <td>210</td>\n",
       "      <td>62</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>215000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>526350040</td>\n",
       "      <td>20</td>\n",
       "      <td>RH</td>\n",
       "      <td>80.0</td>\n",
       "      <td>11622</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Feedr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>1961</td>\n",
       "      <td>1961</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>Rec</td>\n",
       "      <td>468.0</td>\n",
       "      <td>LwQ</td>\n",
       "      <td>144.0</td>\n",
       "      <td>270.0</td>\n",
       "      <td>882.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>896</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>896</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>5</td>\n",
       "      <td>Typ</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1961.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>1.0</td>\n",
       "      <td>730.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>140</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>120</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>MnPrv</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>105000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>526351010</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>81.0</td>\n",
       "      <td>14267</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>1958</td>\n",
       "      <td>1958</td>\n",
       "      <td>Hip</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>108.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>ALQ</td>\n",
       "      <td>923.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>406.0</td>\n",
       "      <td>1329.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>1329</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1329</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Gd</td>\n",
       "      <td>6</td>\n",
       "      <td>Typ</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1958.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>1.0</td>\n",
       "      <td>312.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>393</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gar2</td>\n",
       "      <td>12500</td>\n",
       "      <td>6</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>172000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>526353030</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>93.0</td>\n",
       "      <td>11160</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>1968</td>\n",
       "      <td>1968</td>\n",
       "      <td>Hip</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>ALQ</td>\n",
       "      <td>1065.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1045.0</td>\n",
       "      <td>2110.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>Ex</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>2110</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2110</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Ex</td>\n",
       "      <td>8</td>\n",
       "      <td>Typ</td>\n",
       "      <td>2</td>\n",
       "      <td>TA</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1968.0</td>\n",
       "      <td>Fin</td>\n",
       "      <td>2.0</td>\n",
       "      <td>522.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>244000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>527105010</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>74.0</td>\n",
       "      <td>13830</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Gilbert</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>2Story</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1997</td>\n",
       "      <td>1998</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>PConc</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>GLQ</td>\n",
       "      <td>791.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>928.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>Gd</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>928</td>\n",
       "      <td>701</td>\n",
       "      <td>0</td>\n",
       "      <td>1629</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>6</td>\n",
       "      <td>Typ</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1997.0</td>\n",
       "      <td>Fin</td>\n",
       "      <td>2.0</td>\n",
       "      <td>482.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>212</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>MnPrv</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>189900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Order        PID  MS SubClass MS Zoning  Lot Frontage  Lot Area Street  \\\n",
       "0      1  526301100           20        RL         141.0     31770   Pave   \n",
       "1      2  526350040           20        RH          80.0     11622   Pave   \n",
       "2      3  526351010           20        RL          81.0     14267   Pave   \n",
       "3      4  526353030           20        RL          93.0     11160   Pave   \n",
       "4      5  527105010           60        RL          74.0     13830   Pave   \n",
       "\n",
       "  Alley Lot Shape Land Contour Utilities Lot Config Land Slope Neighborhood  \\\n",
       "0   NaN       IR1          Lvl    AllPub     Corner        Gtl        NAmes   \n",
       "1   NaN       Reg          Lvl    AllPub     Inside        Gtl        NAmes   \n",
       "2   NaN       IR1          Lvl    AllPub     Corner        Gtl        NAmes   \n",
       "3   NaN       Reg          Lvl    AllPub     Corner        Gtl        NAmes   \n",
       "4   NaN       IR1          Lvl    AllPub     Inside        Gtl      Gilbert   \n",
       "\n",
       "  Condition 1 Condition 2 Bldg Type House Style  Overall Qual  Overall Cond  \\\n",
       "0        Norm        Norm      1Fam      1Story             6             5   \n",
       "1       Feedr        Norm      1Fam      1Story             5             6   \n",
       "2        Norm        Norm      1Fam      1Story             6             6   \n",
       "3        Norm        Norm      1Fam      1Story             7             5   \n",
       "4        Norm        Norm      1Fam      2Story             5             5   \n",
       "\n",
       "   Year Built  Year Remod/Add Roof Style Roof Matl Exterior 1st Exterior 2nd  \\\n",
       "0        1960            1960        Hip   CompShg      BrkFace      Plywood   \n",
       "1        1961            1961      Gable   CompShg      VinylSd      VinylSd   \n",
       "2        1958            1958        Hip   CompShg      Wd Sdng      Wd Sdng   \n",
       "3        1968            1968        Hip   CompShg      BrkFace      BrkFace   \n",
       "4        1997            1998      Gable   CompShg      VinylSd      VinylSd   \n",
       "\n",
       "  Mas Vnr Type  Mas Vnr Area Exter Qual Exter Cond Foundation Bsmt Qual  \\\n",
       "0        Stone         112.0         TA         TA     CBlock        TA   \n",
       "1         None           0.0         TA         TA     CBlock        TA   \n",
       "2      BrkFace         108.0         TA         TA     CBlock        TA   \n",
       "3         None           0.0         Gd         TA     CBlock        TA   \n",
       "4         None           0.0         TA         TA      PConc        Gd   \n",
       "\n",
       "  Bsmt Cond Bsmt Exposure BsmtFin Type 1  BsmtFin SF 1 BsmtFin Type 2  \\\n",
       "0        Gd            Gd            BLQ         639.0            Unf   \n",
       "1        TA            No            Rec         468.0            LwQ   \n",
       "2        TA            No            ALQ         923.0            Unf   \n",
       "3        TA            No            ALQ        1065.0            Unf   \n",
       "4        TA            No            GLQ         791.0            Unf   \n",
       "\n",
       "   BsmtFin SF 2  Bsmt Unf SF  Total Bsmt SF Heating Heating QC Central Air  \\\n",
       "0           0.0        441.0         1080.0    GasA         Fa           Y   \n",
       "1         144.0        270.0          882.0    GasA         TA           Y   \n",
       "2           0.0        406.0         1329.0    GasA         TA           Y   \n",
       "3           0.0       1045.0         2110.0    GasA         Ex           Y   \n",
       "4           0.0        137.0          928.0    GasA         Gd           Y   \n",
       "\n",
       "  Electrical  1st Flr SF  2nd Flr SF  Low Qual Fin SF  Gr Liv Area  \\\n",
       "0      SBrkr        1656           0                0         1656   \n",
       "1      SBrkr         896           0                0          896   \n",
       "2      SBrkr        1329           0                0         1329   \n",
       "3      SBrkr        2110           0                0         2110   \n",
       "4      SBrkr         928         701                0         1629   \n",
       "\n",
       "   Bsmt Full Bath  Bsmt Half Bath  Full Bath  Half Bath  Bedroom AbvGr  \\\n",
       "0             1.0             0.0          1          0              3   \n",
       "1             0.0             0.0          1          0              2   \n",
       "2             0.0             0.0          1          1              3   \n",
       "3             1.0             0.0          2          1              3   \n",
       "4             0.0             0.0          2          1              3   \n",
       "\n",
       "   Kitchen AbvGr Kitchen Qual  TotRms AbvGrd Functional  Fireplaces  \\\n",
       "0              1           TA              7        Typ           2   \n",
       "1              1           TA              5        Typ           0   \n",
       "2              1           Gd              6        Typ           0   \n",
       "3              1           Ex              8        Typ           2   \n",
       "4              1           TA              6        Typ           1   \n",
       "\n",
       "  Fireplace Qu Garage Type  Garage Yr Blt Garage Finish  Garage Cars  \\\n",
       "0           Gd      Attchd         1960.0           Fin          2.0   \n",
       "1          NaN      Attchd         1961.0           Unf          1.0   \n",
       "2          NaN      Attchd         1958.0           Unf          1.0   \n",
       "3           TA      Attchd         1968.0           Fin          2.0   \n",
       "4           TA      Attchd         1997.0           Fin          2.0   \n",
       "\n",
       "   Garage Area Garage Qual Garage Cond Paved Drive  Wood Deck SF  \\\n",
       "0        528.0          TA          TA           P           210   \n",
       "1        730.0          TA          TA           Y           140   \n",
       "2        312.0          TA          TA           Y           393   \n",
       "3        522.0          TA          TA           Y             0   \n",
       "4        482.0          TA          TA           Y           212   \n",
       "\n",
       "   Open Porch SF  Enclosed Porch  3Ssn Porch  Screen Porch  Pool Area Pool QC  \\\n",
       "0             62               0           0             0          0     NaN   \n",
       "1              0               0           0           120          0     NaN   \n",
       "2             36               0           0             0          0     NaN   \n",
       "3              0               0           0             0          0     NaN   \n",
       "4             34               0           0             0          0     NaN   \n",
       "\n",
       "   Fence Misc Feature  Misc Val  Mo Sold  Yr Sold Sale Type Sale Condition  \\\n",
       "0    NaN          NaN         0        5     2010       WD          Normal   \n",
       "1  MnPrv          NaN         0        6     2010       WD          Normal   \n",
       "2    NaN         Gar2     12500        6     2010       WD          Normal   \n",
       "3    NaN          NaN         0        4     2010       WD          Normal   \n",
       "4  MnPrv          NaN         0        3     2010       WD          Normal   \n",
       "\n",
       "   SalePrice  \n",
       "0     215000  \n",
       "1     105000  \n",
       "2     172000  \n",
       "3     244000  \n",
       "4     189900  "
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn import linear_model\n",
    "import seaborn as sns\n",
    "\n",
    "pd.options.display.max_columns = 999\n",
    "\n",
    "houses = pd.read_csv('AmesHousing.tsv', delimiter = '\\t')\n",
    "\n",
    "houses.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the necessary functions\n",
    "\n",
    "Now, we'll create a pipeline of functions to allow us to quickly iterate on different models. \n",
    "\n",
    "The below functions will be modified to allow for various feature selections and transformations as we proceed in this project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def transform_features(df): # For now this just returns the same df\n",
    "    return df\n",
    "\n",
    "def select_features(df): # For now this only returns the specified 2 cols\n",
    "    return df[[\"Gr Liv Area\", \"SalePrice\"]]\n",
    "\n",
    "def train_and_test(df):\n",
    "    train = df.iloc[:1460]\n",
    "    test = df.iloc[1460:]\n",
    "    \n",
    "    # Selecting only numeric columns, other than SalePrice    \n",
    "    numeric_train = train.select_dtypes(include=['integer', 'float'])\n",
    "    numeric_test = test.select_dtypes(include=['integer', 'float'])    \n",
    "    \n",
    "    numeric_features = numeric_train.columns.drop('SalePrice')\n",
    "    \n",
    "    # Instantiating model:\n",
    "    lr = linear_model.LinearRegression()\n",
    "    \n",
    "    # Training:\n",
    "    lr.fit(numeric_train[numeric_features], numeric_train['SalePrice'])\n",
    "    \n",
    "    # Predicting:\n",
    "    train_predictions = lr.predict(numeric_test[numeric_features])\n",
    "    \n",
    "    # RMSE:\n",
    "    rmse = mean_squared_error(train_predictions, numeric_test['SalePrice']) ** 0.5\n",
    "    \n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll run the above functions on the dataframe we read in, `houses`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "57088.25161263909\n"
     ]
    }
   ],
   "source": [
    "transformed_houses = transform_features(houses)\n",
    "\n",
    "features_selected = select_features(transformed_houses)\n",
    "\n",
    "rmse_houses = train_and_test(features_selected)\n",
    "\n",
    "print(rmse_houses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that using only the `Gr Liv Area` column, and not transforming the `houses` dataframe at all, results in an rmse of 57088.\n",
    "\n",
    "Now, we'll begin modifying the dataframe and being more careful of feature selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering\n",
    "\n",
    "We'll make the following modifications to the `transform features` function to handle missing values:\n",
    "\n",
    "- Any column with more than 15% missing values will be dropped.\n",
    "- For text columns in particular, we'll drop columns with any missing values at all.\n",
    "- For numeric columns, we'll impute missing values as being the average of that column.\n",
    "\n",
    "### Handling Missing Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Order                0\n",
       "PID                  0\n",
       "MS SubClass          0\n",
       "MS Zoning            0\n",
       "Lot Frontage       490\n",
       "Lot Area             0\n",
       "Street               0\n",
       "Alley             2732\n",
       "Lot Shape            0\n",
       "Land Contour         0\n",
       "Utilities            0\n",
       "Lot Config           0\n",
       "Land Slope           0\n",
       "Neighborhood         0\n",
       "Condition 1          0\n",
       "Condition 2          0\n",
       "Bldg Type            0\n",
       "House Style          0\n",
       "Overall Qual         0\n",
       "Overall Cond         0\n",
       "Year Built           0\n",
       "Year Remod/Add       0\n",
       "Roof Style           0\n",
       "Roof Matl            0\n",
       "Exterior 1st         0\n",
       "Exterior 2nd         0\n",
       "Mas Vnr Type        23\n",
       "Mas Vnr Area        23\n",
       "Exter Qual           0\n",
       "Exter Cond           0\n",
       "                  ... \n",
       "Bedroom AbvGr        0\n",
       "Kitchen AbvGr        0\n",
       "Kitchen Qual         0\n",
       "TotRms AbvGrd        0\n",
       "Functional           0\n",
       "Fireplaces           0\n",
       "Fireplace Qu      1422\n",
       "Garage Type        157\n",
       "Garage Yr Blt      159\n",
       "Garage Finish      159\n",
       "Garage Cars          1\n",
       "Garage Area          1\n",
       "Garage Qual        159\n",
       "Garage Cond        159\n",
       "Paved Drive          0\n",
       "Wood Deck SF         0\n",
       "Open Porch SF        0\n",
       "Enclosed Porch       0\n",
       "3Ssn Porch           0\n",
       "Screen Porch         0\n",
       "Pool Area            0\n",
       "Pool QC           2917\n",
       "Fence             2358\n",
       "Misc Feature      2824\n",
       "Misc Val             0\n",
       "Mo Sold              0\n",
       "Yr Sold              0\n",
       "Sale Type            0\n",
       "Sale Condition       0\n",
       "SalePrice            0\n",
       "Length: 82, dtype: int64"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "houses_2 = houses.copy()\n",
    "\n",
    "houses_2.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since there are 2930 rows in `houses_2`, we'll remove columns where there are more than 439 missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "76"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "houses_2_null_count = houses_2.isnull().sum()\n",
    "houses_2_null_count = houses_2_null_count[houses_2_null_count < 440]\n",
    "\n",
    "len(houses_2_null_count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2930, 76)"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "houses_2_filtered = houses_2[houses_2_null_count.index]\n",
    "\n",
    "houses_2_filtered.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see we're now down to 76 columns. Next, we'll handle numeric columns with missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2930, 38)"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric_filtered = houses_2_filtered.select_dtypes(include=['float', 'int'])\n",
    "\n",
    "numeric_filtered.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric_filtered = numeric_filtered.fillna(numeric_filtered.mean())\n",
    "\n",
    "# Using .sum() twice to tally up null values in the entire dataframe.\n",
    "\n",
    "numeric_filtered.isnull().sum().sum() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we'll examine the text columns more closely and drop columns with any missing values at all."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2930, 38)"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_filtered = houses_2_filtered.select_dtypes(include=['object'])\n",
    "\n",
    "text_filtered.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2930, 27)"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_filtered = text_filtered.dropna(axis=1)\n",
    "\n",
    "text_filtered.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After dropping columns with missing values, we're left with 27 text columns. We'll verify that there aren't any missing values left. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_filtered.isnull().sum().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now concatenate the `text_filtered` and `numeric_filtered` dataframes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2930, 65)"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "houses_3 = pd.concat([text_filtered, numeric_filtered], axis = 1)\n",
    "\n",
    "houses_3.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating new features\n",
    "\n",
    "From the existing columns in the dataframe, we see that we can use the `Yr Sold`, `Year Built`, and `Year Remod/Add` columns to create new columns that reflect:\n",
    "\n",
    "1. The years until a house was sold, and \n",
    "2. The years until a house was remodelled."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2930, 67)"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "houses_3['Years Before Sale'] = houses_3['Yr Sold'] - houses_3['Year Built']\n",
    "houses_3['Years Since Remod'] = houses_3['Year Remod/Add'] - houses_3['Year Built']\n",
    "\n",
    "houses_3.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the year a house was bought or remodelled cannot be before the year it was built, we'll drop rows in which the values in the `Years Before Sale` or `Years Since Remod` columns are negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2928, 67)"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "houses_3 = houses_3[houses_3['Years Before Sale'] >= 0]\n",
    "houses_3 = houses_3[houses_3['Years Since Remod'] >= 0]\n",
    "\n",
    "houses_3.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that a total of 2 rows was lost from removing the incorrect values, which is not a significant amount."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dropping columns that aren't needed\n",
    "\n",
    "We'll drop columns that are either:\n",
    "\n",
    "1. Not useful, or\n",
    "2. Leak Data about the final sale.\n",
    "\n",
    "The `PID` column and `Order` columns merely identify each unique house and don't add anything meaningful about its features. As such, we'll drop these."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "houses_3 = houses_3.drop([\"PID\", \"Order\"], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the `Mo Sold`, `Sale Condition`, `Sale Type`, and `Yr Sold` columns leak data about the final sale, these columns will be dropped too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2928, 61)"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "houses_3 = houses_3.drop([\"Mo Sold\", \"Sale Condition\", \"Sale Type\", \"Yr Sold\"], axis=1)\n",
    "\n",
    "houses_3.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Updating `transform_features` function\n",
    "\n",
    "We'll now update the `transform_features` function to include the above code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def transform_features(df):\n",
    "    # Removing columns with more than 15% missing values\n",
    "    df_null_count = df.isnull().sum()\n",
    "    df_null_count = df_null_count[df_null_count < 440] \n",
    "    df_filtered = df[df_null_count.index]\n",
    "    \n",
    "    # Modifying Numeric Columns\n",
    "    numeric_filtered = df_filtered.select_dtypes(include=['float', 'int'])\n",
    "    numeric_filtered = numeric_filtered.fillna(numeric_filtered.mean())\n",
    "    \n",
    "    # Modifying Text Columns\n",
    "    text_filtered = df_filtered.select_dtypes(include=['object'])\n",
    "    text_filtered = text_filtered.dropna(axis=1)\n",
    "    \n",
    "    # Concatenating Numeric and Text columns\n",
    "    df_new = pd.concat([text_filtered, numeric_filtered], axis = 1)\n",
    "    \n",
    "    # Creating new columns, and dropping the 2 rows that had negative values there\n",
    "    df_new['Years Before Sale'] = df_new['Yr Sold'] - df_new['Year Built']\n",
    "    df_new['Years Since Remod'] = df_new['Year Remod/Add'] - df_new['Year Built']\n",
    "    df_new = df_new[df_new['Years Before Sale'] >= 0]\n",
    "    df_new = df_new[df_new['Years Since Remod'] >= 0]\n",
    "    \n",
    "    # Dropping unnecessary columns:\n",
    "    df_new = df_new.drop([\"PID\", \"Order\"], axis=1)\n",
    "    df_new = df_new.drop([\"Mo Sold\", \"Sale Condition\", \"Sale Type\", \"Yr Sold\"], axis=1)\n",
    "\n",
    "    return df_new\n",
    "\n",
    "def select_features(df):\n",
    "    return df[[\"Gr Liv Area\", \"SalePrice\"]]\n",
    "\n",
    "def train_and_test(df):\n",
    "    train = df.iloc[:1460]\n",
    "    test = df.iloc[1460:]\n",
    "    \n",
    "    # Selecting only numeric columns, other than SalePrice    \n",
    "    numeric_train = train.select_dtypes(include=['integer', 'float'])\n",
    "    numeric_test = test.select_dtypes(include=['integer', 'float'])    \n",
    "    \n",
    "    numeric_features = numeric_train.columns.drop('SalePrice')\n",
    "    \n",
    "    # Instantiating model:\n",
    "    lr = linear_model.LinearRegression()\n",
    "    \n",
    "    # Training:\n",
    "    lr.fit(numeric_train[numeric_features], numeric_train['SalePrice'])\n",
    "    \n",
    "    # Predicting:\n",
    "    train_predictions = lr.predict(numeric_test[numeric_features])\n",
    "    \n",
    "    # RMSE:\n",
    "    rmse = mean_squared_error(train_predictions, numeric_test['SalePrice']) ** 0.5\n",
    "    \n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the modified `transform_features` function, we'll now check the rmse value with the 3 functions as they are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "56072.52569017749"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"AmesHousing.tsv\", delimiter=\"\\t\")\n",
    "\n",
    "transform_df = transform_features(df)\n",
    "filtered_df = select_features(transform_df)\n",
    "rmse = train_and_test(filtered_df)\n",
    "\n",
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We see that there was a marginal improvement in the rmse from transforming the dataframe.\n",
    "\n",
    "Now, we'll look into how to improve the feature selection process to improve the predictive accuracy.\n",
    "\n",
    "## Feature Selection\n",
    "\n",
    "To begin with, we'll generate a correlation heatmap matrix of numerical features in the training data set. We'll use the `transform_df` that was created just above by the modified `transform_features` function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2928, 36)\n"
     ]
    }
   ],
   "source": [
    "df_numeric = transform_df.select_dtypes(include=['int', 'float'])\n",
    "\n",
    "print(df_numeric.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MS SubClass</th>\n",
       "      <th>Lot Area</th>\n",
       "      <th>Overall Qual</th>\n",
       "      <th>Overall Cond</th>\n",
       "      <th>Year Built</th>\n",
       "      <th>Year Remod/Add</th>\n",
       "      <th>Mas Vnr Area</th>\n",
       "      <th>BsmtFin SF 1</th>\n",
       "      <th>BsmtFin SF 2</th>\n",
       "      <th>Bsmt Unf SF</th>\n",
       "      <th>Total Bsmt SF</th>\n",
       "      <th>1st Flr SF</th>\n",
       "      <th>2nd Flr SF</th>\n",
       "      <th>Low Qual Fin SF</th>\n",
       "      <th>Gr Liv Area</th>\n",
       "      <th>Bsmt Full Bath</th>\n",
       "      <th>Bsmt Half Bath</th>\n",
       "      <th>Full Bath</th>\n",
       "      <th>Half Bath</th>\n",
       "      <th>Bedroom AbvGr</th>\n",
       "      <th>Kitchen AbvGr</th>\n",
       "      <th>TotRms AbvGrd</th>\n",
       "      <th>Fireplaces</th>\n",
       "      <th>Garage Yr Blt</th>\n",
       "      <th>Garage Cars</th>\n",
       "      <th>Garage Area</th>\n",
       "      <th>Wood Deck SF</th>\n",
       "      <th>Open Porch SF</th>\n",
       "      <th>Enclosed Porch</th>\n",
       "      <th>3Ssn Porch</th>\n",
       "      <th>Screen Porch</th>\n",
       "      <th>Pool Area</th>\n",
       "      <th>Misc Val</th>\n",
       "      <th>SalePrice</th>\n",
       "      <th>Years Before Sale</th>\n",
       "      <th>Years Since Remod</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20</td>\n",
       "      <td>31770</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>1960</td>\n",
       "      <td>1960</td>\n",
       "      <td>112.0</td>\n",
       "      <td>639.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>441.0</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>1656</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1656</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>1960.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>528.0</td>\n",
       "      <td>210</td>\n",
       "      <td>62</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>215000</td>\n",
       "      <td>50</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>20</td>\n",
       "      <td>11622</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>1961</td>\n",
       "      <td>1961</td>\n",
       "      <td>0.0</td>\n",
       "      <td>468.0</td>\n",
       "      <td>144.0</td>\n",
       "      <td>270.0</td>\n",
       "      <td>882.0</td>\n",
       "      <td>896</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>896</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>1961.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>730.0</td>\n",
       "      <td>140</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>120</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>105000</td>\n",
       "      <td>49</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20</td>\n",
       "      <td>14267</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>1958</td>\n",
       "      <td>1958</td>\n",
       "      <td>108.0</td>\n",
       "      <td>923.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>406.0</td>\n",
       "      <td>1329.0</td>\n",
       "      <td>1329</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1329</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1958.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>312.0</td>\n",
       "      <td>393</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12500</td>\n",
       "      <td>172000</td>\n",
       "      <td>52</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   MS SubClass  Lot Area  Overall Qual  Overall Cond  Year Built  \\\n",
       "0           20     31770             6             5        1960   \n",
       "1           20     11622             5             6        1961   \n",
       "2           20     14267             6             6        1958   \n",
       "\n",
       "   Year Remod/Add  Mas Vnr Area  BsmtFin SF 1  BsmtFin SF 2  Bsmt Unf SF  \\\n",
       "0            1960         112.0         639.0           0.0        441.0   \n",
       "1            1961           0.0         468.0         144.0        270.0   \n",
       "2            1958         108.0         923.0           0.0        406.0   \n",
       "\n",
       "   Total Bsmt SF  1st Flr SF  2nd Flr SF  Low Qual Fin SF  Gr Liv Area  \\\n",
       "0         1080.0        1656           0                0         1656   \n",
       "1          882.0         896           0                0          896   \n",
       "2         1329.0        1329           0                0         1329   \n",
       "\n",
       "   Bsmt Full Bath  Bsmt Half Bath  Full Bath  Half Bath  Bedroom AbvGr  \\\n",
       "0             1.0             0.0          1          0              3   \n",
       "1             0.0             0.0          1          0              2   \n",
       "2             0.0             0.0          1          1              3   \n",
       "\n",
       "   Kitchen AbvGr  TotRms AbvGrd  Fireplaces  Garage Yr Blt  Garage Cars  \\\n",
       "0              1              7           2         1960.0          2.0   \n",
       "1              1              5           0         1961.0          1.0   \n",
       "2              1              6           0         1958.0          1.0   \n",
       "\n",
       "   Garage Area  Wood Deck SF  Open Porch SF  Enclosed Porch  3Ssn Porch  \\\n",
       "0        528.0           210             62               0           0   \n",
       "1        730.0           140              0               0           0   \n",
       "2        312.0           393             36               0           0   \n",
       "\n",
       "   Screen Porch  Pool Area  Misc Val  SalePrice  Years Before Sale  \\\n",
       "0             0          0         0     215000                 50   \n",
       "1           120          0         0     105000                 49   \n",
       "2             0          0     12500     172000                 52   \n",
       "\n",
       "   Years Since Remod  \n",
       "0                  0  \n",
       "1                  0  \n",
       "2                  0  "
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_numeric.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selecting numeric features that correlate strongly with target column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BsmtFin SF 2         0.005922\n",
       "Misc Val             0.019295\n",
       "3Ssn Porch           0.032236\n",
       "Bsmt Half Bath       0.035933\n",
       "Low Qual Fin SF      0.037649\n",
       "Pool Area            0.068411\n",
       "MS SubClass          0.085020\n",
       "Overall Cond         0.101652\n",
       "Screen Porch         0.112187\n",
       "Kitchen AbvGr        0.119795\n",
       "Enclosed Porch       0.128756\n",
       "Bedroom AbvGr        0.143939\n",
       "Bsmt Unf SF          0.182944\n",
       "Years Since Remod    0.240133\n",
       "Lot Area             0.267122\n",
       "2nd Flr SF           0.269518\n",
       "Bsmt Full Bath       0.275891\n",
       "Half Bath            0.285217\n",
       "Open Porch SF        0.315141\n",
       "Wood Deck SF         0.327790\n",
       "BsmtFin SF 1         0.437376\n",
       "Fireplaces           0.475073\n",
       "TotRms AbvGrd        0.497966\n",
       "Mas Vnr Area         0.509181\n",
       "Garage Yr Blt        0.510791\n",
       "Year Remod/Add       0.533076\n",
       "Full Bath            0.545641\n",
       "Year Built           0.558560\n",
       "Years Before Sale    0.559042\n",
       "1st Flr SF           0.632531\n",
       "Total Bsmt SF        0.641287\n",
       "Garage Area          0.641433\n",
       "Garage Cars          0.648121\n",
       "Gr Liv Area          0.712893\n",
       "Overall Qual         0.800279\n",
       "SalePrice            1.000000\n",
       "Name: SalePrice, dtype: float64"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_numeric.corr()['SalePrice'].sort_values()\n",
    "\n",
    "# absolute should be applied, because we are interested in the strength of correlation\n",
    "\n",
    "df_corr = df_numeric.corr()['SalePrice'].abs().sort_values()\n",
    "\n",
    "df_corr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll include only columns with a correlation greater than 0.4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>BsmtFin SF 1</th>\n",
       "      <th>Fireplaces</th>\n",
       "      <th>TotRms AbvGrd</th>\n",
       "      <th>Mas Vnr Area</th>\n",
       "      <th>Garage Yr Blt</th>\n",
       "      <th>Year Remod/Add</th>\n",
       "      <th>Full Bath</th>\n",
       "      <th>Year Built</th>\n",
       "      <th>Years Before Sale</th>\n",
       "      <th>1st Flr SF</th>\n",
       "      <th>Total Bsmt SF</th>\n",
       "      <th>Garage Area</th>\n",
       "      <th>Garage Cars</th>\n",
       "      <th>Gr Liv Area</th>\n",
       "      <th>Overall Qual</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>639.0</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>112.0</td>\n",
       "      <td>1960.0</td>\n",
       "      <td>1960</td>\n",
       "      <td>1</td>\n",
       "      <td>1960</td>\n",
       "      <td>50</td>\n",
       "      <td>1656</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>528.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1656</td>\n",
       "      <td>6</td>\n",
       "      <td>215000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>468.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1961.0</td>\n",
       "      <td>1961</td>\n",
       "      <td>1</td>\n",
       "      <td>1961</td>\n",
       "      <td>49</td>\n",
       "      <td>896</td>\n",
       "      <td>882.0</td>\n",
       "      <td>730.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>896</td>\n",
       "      <td>5</td>\n",
       "      <td>105000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>923.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>108.0</td>\n",
       "      <td>1958.0</td>\n",
       "      <td>1958</td>\n",
       "      <td>1</td>\n",
       "      <td>1958</td>\n",
       "      <td>52</td>\n",
       "      <td>1329</td>\n",
       "      <td>1329.0</td>\n",
       "      <td>312.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1329</td>\n",
       "      <td>6</td>\n",
       "      <td>172000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   BsmtFin SF 1  Fireplaces  TotRms AbvGrd  Mas Vnr Area  Garage Yr Blt  \\\n",
       "0         639.0           2              7         112.0         1960.0   \n",
       "1         468.0           0              5           0.0         1961.0   \n",
       "2         923.0           0              6         108.0         1958.0   \n",
       "\n",
       "   Year Remod/Add  Full Bath  Year Built  Years Before Sale  1st Flr SF  \\\n",
       "0            1960          1        1960                 50        1656   \n",
       "1            1961          1        1961                 49         896   \n",
       "2            1958          1        1958                 52        1329   \n",
       "\n",
       "   Total Bsmt SF  Garage Area  Garage Cars  Gr Liv Area  Overall Qual  \\\n",
       "0         1080.0        528.0          2.0         1656             6   \n",
       "1          882.0        730.0          1.0          896             5   \n",
       "2         1329.0        312.0          1.0         1329             6   \n",
       "\n",
       "   SalePrice  \n",
       "0     215000  \n",
       "1     105000  \n",
       "2     172000  "
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strong_corr = df_corr[df_corr > 0.4]\n",
    "\n",
    "transform_df = transform_df[strong_corr.index]\n",
    "\n",
    "transform_df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Identifying collinearity between features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f5e0e722630>"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbQAAAFMCAYAAACj9UkyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8TNf/+PHXTJIhEkFstVWLanzsS4nWmtgjYSKx77RV\nVKu2agmNotRatVRp0S2VRqISFLGVqhBfUktrKaFBEEIikkky8/sjP7dGEhnuREb6fj4e83h07pz7\nvmemrbdz7rnvozGZTCaEEEKIZ5y2oDsghBBCWIMkNCGEEIWCJDQhhBCFgiQ0IYQQhYIkNCGEEIWC\nfUF3QORshOYF1TE+2/uJqvNNDTqp7oMmZpvqGPbPvaA6hknnqOr8TKfSqvvAXwdUh9jl2lLV+U13\nLlDdh6SL8apjlG/fVnUMuxLq/p1oS5ZV3YeMa5dUx0j5I1p1jJJvzFId43H+zFlhuqD6evlBRmhC\nCCEKBRmhCSGEwE5T0D1Qr0ATWq1atXBzc8NoNGJnZ0dAQAANGjRQFfPPP/8kPj6e1q1bAxAaGsrc\nuXN57rnnMJlMuLm58cknn/DZZ5/xyiuv0Lx5c4vipqamMmXKFP766y8AXFxcWLVqFY6Ojsr3MJlM\naDQali5dSsWKFc3OHz58OMeOHaNx48asWLFC1XcUQghrs9M8+xmtQBOao6MjoaGhAOzbt4/58+fz\nzTffqIp56tQpjh8/riQ0AC8vL6ZMmWLWbsyYMY8Vd926dZQtW5Z58+YBcOHCBezts36+B79HboYP\nH05qaipBQUGPdV0hhHgadNpnP6EV6D20B6tuJSUlUaJECQCuX79O//790ev1eHt7Ex2dddO0YcOG\nzJ07l65duzJ06FCOHTvGgAEDaN++Pbt27SI9PZ3PPvuMLVu2oNfr2bJlS7br3Dd58mS2bctasODh\n4cGSJUvw9fXFx8eH8+fPZ2t/7do1ypcvr7x/4YUXcHBwyDX+w9zd3SlWrJilP40QQjxVdhrLX7aq\nQEdoaWlp6PV6UlNTuXHjBmvXrgUgPDycli1b8uabb2Iymbh37x4A9+7d49VXX2XixImMHj2azz77\njLVr13L69Gnef/992rZty5gxYzhx4oQyIgsNDWXz5s0cOXIEgIEDB6LX67P1xdXVlQ0bNvD999+z\nevVqPv74Y7PP/fz8GDp0KFu3bsXd3R29Xk/VqlXNvofJZKJKlSosWbIk334zIYTIDzLlqFLRokWV\nqbqjR48yceJEwsPDqVu3Lh9++CHp6em0a9cONzc3AHQ6HS1atACgZs2aFClSBK1Wy8svv8zly5dz\nvU5OU44Pa9++PQB16tRhx44d2T53c3MjMjKS/fv3s3//fvz9/QkKCqJatWpm30MIIZ5FtjzyspTN\nLNtv0KABt27d4ubNmzRp0oRvv/2W8uXL8/7777Nx40YA5Z4VgFarRafTAaDRaMjIyFB1/fuxtFpt\nrrEcHR1p164d06ZNw9vbm7179wKWTTkKIYQts9NoLH7ZKpu5h3bu3DmMRiOlSpXi8uXLlC5dGn9/\nf/z9/Tl58mS29rnFcnJyIjk52ep9PXLkCHfu3AHAYDBw9uxZKlWq9FgxJPEJIWyV9jFetqpApxwN\nBoNy7wlgzpw5aDQaoqKiWL16Nfb29jg5OTF37lwgaySWm/ufNWvWjJUrV6LX63njjTcs6sej4t53\n8eJFpk+fDmQlptatWyvTlJac369fP86fP09KSgpt2rRh5syZvPbaaxb1Twgh8pstj7wspZENPm2T\nlL76l5S++peUvvqXlL76lzVKX33qXNPithOST6u+Xn6QSiFCCCEKxaIQSWhCCCEKxZSjJDQhhBAy\nQhP5R+39L4Axrd5Xdf7iXZmq+6BxdFIdI9PlOfX9MNxVdb7d5VOq+0CF6qpDVHUsqup8eyd15wOU\na21Z/dNH0TjoVMfISLiq6vzk/TtV98GxbCnVMQx31P23aS0yQhNCCFEoFIYRmi0/UpCrWrVqodfr\n6d69O3q9nsuXL3P8+HFmzpxptWt4eHiQmJhotXhCCGHLpJZjAcmpun3FihWpU6dOtraZmZnY2dk9\n9jUsebZMCCEKi8JQbf+ZTGg5PToXFRXFV199xYoVK/j888+5ePEily5domLFinz66afMmzePQ4cO\nYTAY6NevHz179iQqKorPPvsMJycnYmNjcXd3N3t4+r5Ro0Zx9epVDAYDAwcOxN/fH4C9e/eyaNEi\npcLJ119/zb1795gxYwZnz54lIyOD0aNH4+HhwdmzZ5k8eTIZGRkYjUaWLFnC888//1R+LyGEyIu1\n76Ht3buXWbNmYTKZ6NGjR7ZCF7du3WLChAlcv34do9HIkCFD8PX1VXXNZzKhWVLd/ty5c/zwww/o\ndDrWr1+Pi4sLwcHBGAwG+vTpo1Tp+OOPP9i8eTMVK1Zk2LBhbNu2jQ4dOpjFmj17Ni4uLqSlpeHn\n50eHDh3IzMwkICCA77//nooVKyplsVasWEHz5s2ZNWsWSUlJ+Pn58eqrrxIUFMSgQYPo2rWrktSE\nEMJWWHMq0Wg0MmPGDNasWUO5cuXw8/PD09OT6tX/XRj13XffUatWLVatWsXNmzfp3LkzPj4+ZjV7\nH9czmdAsqW7v4eGhFBzet28fp0+fZuvWrQAkJycTGxuLvb099erVU2oyenl5ER0dnS2hrV27VqnA\nf/XqVWJjY0lISOCVV15RdqZ2cXFRrrVz505Wr14NQHp6OpcvX6ZBgwasWLGCq1ev0r59e2XrGSGE\nsAXWHKHFxMRQtWpVsz9bIyMjzRJamTJlOH06q+LI3bt3KVmypKpkBs9oQrPEw5tpTp06NVvtxKio\nqGznPXzvLCoqit9//53g4GB0Oh0DBgwgLS0NyL3Y8JIlS3jhhRfMjlWrVo369euze/du3njjDQID\nA2nWrNnjfi0hhMgX1hyhxcfHU6FCBeV9+fLl+eOPP8za9OzZk0GDBtGiRQtSUlJYuHCh6us+k6sc\nH7f8ZIsWLfj++++VbWEuXLhAamoqkDXlGBcXh9FoZPPmzTRp0sTs3KSkJFxcXNDpdJw7d45jx44B\nUL9+faKjo4mLiwPg9u3byrW++eYb5fxTp7KeX7p06RJVqlRhwIABeHh48Ndffz3BNxdCiPzxtLeP\n+eKLL3Bzc2Pfvn2EhYURGBjI3bvqnsl7Jkdoj7sC0d/fn7i4OGWnaldXV5YuXQpkbeg5Y8YMZVFI\nu3btzK7RsmVLgoKC8PLy4sUXX6RBgwZKjMDAQEaPHo3JZKJ06dKsXr2akSNHMnPmTLy9vTGZTFSu\nXJkVK1awZcsWfv75Z+zt7SlbtixvvfWWtX4OIYRQTWvFKcfy5cubbbocHx9PuXLlzNocOXKEESNG\nAPD8889TuXJl/v77b+rWrfvE130mE9qRI0eyHWvatClNmzYFYPTo0WafaTQaxo4dy9ixY7Od5+zs\nzIoVK7Idj4yMVP75yy+/zLEfLVu2pGVL8+rnRYoUITAwMFvbN954w+LtbIQQ4mmz01lvwq5u3bpc\nvHiRuLg4ypYtS0REBAsWmO/0UL16dQ4cOEDjxo25ceMGFy5coEqVKqqu+0wmNCGEENalsbNeQrOz\ns2Pq1KkMHToUk8mEn58f1atXJygoCI1GQ69evXjjjTf44IMP8PHxwWQyMWHCBEqWLKnquv/phPbg\nqE4IIf7LNFYuAdKqVStatWpldqx3797KP7u6uuY4O6bGfzqhCSGEyKK15ZpWFpKEZqOssVu02mr5\n77T9UHUflvy+SHUMbept1TE0aepWT/3z3beq+1DhnSmqYxRzUDctZOdYLO9GecUoVS7vRnnIvJ2g\nOoYpVd2/019nbVXdh45LBqmOkZFqUB3DGjTaZ3LRuxlJaEIIIWSEdl9iYiKDBw9Go9Fw/fp1tFot\nrq6uaDQagoODsz39ffv2bbZs2aLMp168eBFvb2+qVatGeno6jRo1ynGloDUEBgaya9cudu3apRxb\ntGgRrq6uDBw40OI4R48eZf78+Vy7dg0nJyfKly/P+PHjzZ6Ez03r1q2JiIjA2dn5ib6DEEJYm7Xv\noRUEqyS0kiVLEhYWBsDnn3+Ok5MTQ4YMybV9YmIiQUFBZjcIq1WrRmhoKJmZmQwePJjIyEg8PT2t\n0T2F0Whkz549VK1alejoaBo3bvxEca5fv857773H4sWLlWcmDh8+zMWLF7MltJyq/UslfyGErbHT\nPf6uJLYm36ccv/zyS37++Wcga4VLv379mD9/PhcuXECv19OyZUv8/PyU9nZ2dtSpU4dLly4BEBwc\nzN69e5X6i8OHD+fu3bts3ryZokWL8uWXX+Ls7MyaNWuU0eDLL7/M3Llzs/XlwIED1KlTB09PT8LD\nw80S2smTJ+nVqxeJiYm8+eab+Pr6MmbMGHr16qWUzJowYQKdOnXi2LFj+Pv7mz0A+GCFkQkTJuDk\n5MSJEydo1qwZQ4cOZfz48Vy/fp2GDRs+dqUTIYTIb5pCsH1Mvt4FjImJISIigpCQEIKCgvjuu+84\nc+YM48eP58UXXyQ0NJT33nsP+LecVWpqKocPH6ZGjRpKnLNnz7JixQqCg4OZN28erq6uhIaGUrt2\nbTZu3AjAqlWr2LhxIxs3biQgICDH/oSHh9OlSxc8PDzYvXu3WcX706dP8+233/LDDz+wePFibt68\nSZcuXdi8eTMABoOBw4cP06pVK86cOcP//ve/R373GzduEBwczPjx41myZAnu7u5s2rSJ1q1bc+3a\ntSf/UYUQIh9o7bQWv2xVvvbsfuV6nU6Hk5MT7dq14/Dhwzm2PX/+PHq9ntdee41KlSrRokUL5TN3\nd3eKFClC6dKlcXZ2pm3btgDUrFlTqaVYs2ZNxo8fz6ZNm3Lc0NNgMLBv3z7atGmDs7MztWrV4rff\nflM+b9euHQ4ODri6uvLKK6/wxx9/0KZNGw4cOEBmZia7d+/G3d0dBweHbLF79OhB586dmTNnjnKs\nU6d/VykePnwYHx8fADw9PXFycnqcn1EIIfKdxk5j8ctW2UyqvX8Pbfv27Zw+fVop6gso28BA1v2n\n++81Gg2ZmVlL01evXk2fPn34448/8Pf3zzatd3/a8v4I7ejRo0RERJjFve/+uUWLFqVRo0bs27eP\nzZs307lzZwBeeuklTpw4obQPCQnh7bffJjk5WTnm6OiY63eVKUchhK2RhJaHxo0bs2PHDgwGA3fv\n3iUyMpImTZrg5OSUrary/T/kXV1dGTNmTLa6X49iMpm4cuUKzZo1Y/z48SQmJnLv3j2zNuHh4cyZ\nM4fIyEh27tzJ9u3b2bt3LwZD1jMgO3bsID09nZs3b3LkyBHl/ljnzp0JCQnh2LFjyr20/v3789NP\nPxETE6PEf/h6D3rllVeU+4g7d+4kJSXF4u8mhBBPQ2GYcszXRSH16tXDy8uLHj16ANC3b19eeukl\nAGrXro2Pjw9t2rTBz8/PbITUqVMnli5dyvHjx7PFzGmFYHp6OuPGjSMlJQWj0ciwYcPM9kNLSUnh\nwIEDzJ49Wznm5ORE/fr12bNnDxqNhpo1a9K/f38SExMZM2YMrq6uQFYB4smTJ9OpUydlKrNcuXIs\nWLCAOXPmcOPGDUqXLk2pUqWUosgP93H06NGMGzeOTZs20ahRo2xVp4UQoqDZqXxo3xZoTDL/ZZPS\nkhLVB4mOyLvNI9hMpZCKeT/blxfVlUJWf6G6D9aoFHJFV17V+WUil6nug8PzNVXHsIVKIVvfUP/v\n1BqVQhL/Oq86RsUpy1XHiPZub3Hbxpu2q75efpBKIUIIIaRSiBBCiMLBlhd7WEoSmhBCCKvuh1ZQ\nJKHZKE3MNvUxHNU972aN+19vu7+rOsbnR9TvmaRxcVV1fgVfveo+GE/uVx0jupzl9zly0lF1DyD9\nwqm8G+XFPvvznI9L46DLu9EjdFjYR3Uf0u/cUR3DmJ6hOoY1yJSjEEKIQkFKXz0hNzc3Jk6cqLzP\nzMzE3d2dESNGqIo7efJk1q9fb3Zsx44dvP7666riAowcOZJevXqpjiOEELbITmdn8ctWFUhCc3R0\n5MyZM8pDzfv376dChQqq43bt2pXw8HCzY5s3b6Zr164Wx7hfeeRBSUlJnDlzhvT0dP755x+LzxNC\niGeFxk5r8ctWFVjPWrVqxe7duwGIiIjAy8tL+SwmJobevXvj6+tLnz59uHDhApBVpNjf3x+9Xk+3\nbt24ePGiWczmzZtz/vx5bty4AWRV7zhw4ADt2rUjLi6OLl26MHXqVLp27cqwYcOUhDpgwABmzZqF\nn58f33zzTba+btu2DQ8PDzp16mRWLmvy5MlMmzaNnj17Mm/ePO7du8cHH3xAz5498fX1ZefOnQDE\nxcXRr18/fH198fX15ejRo1b7HYUQwho0Wq3FL1tVID3TaDR4eXkRHh6OwWDgr7/+on79+srn1atX\n5/vvv2fDhg1mZbCCgoIYNGgQoaGhhISE8Nxzz5nF1Wq1dOzYkS1btgCwa9cumjZtqhQDvnjxIv37\n9yc8PJzixYvzyy+/KOdmZGTw008/MXjw4Gz9jYiIoEuXLnTu3DnbCDA+Pp7169czadIkVqxYQfPm\nzVm/fj1r165lzpw5pKamUqZMGb7++ms2bNjAwoULmTFjhlV+RyGEsBYpfaXC/Ur54eHhtG7d2qxg\nb1JSEpMmTSI2Nhb4dzqvQYMGrFixgqtXr9K+fXuqVq2aLW6XLl2YO3cuAwYMICIigu7duyufVapU\niZdffhnIKr11v1L//fNykpCQQGxsrJJwHRwcOHv2rLK9zYNV9fft28fOnTtZvXo1kFWS6/Lly5Qr\nV47AwEBOnTqFnZ2d8r2EEMJW2PJUoqUK9Bt4eHgwd+7cbPe4Fi9erOwftmLFCtLS0oCse2TLly+n\nSJEivPHGGxw8eDBbzEaNGnH9+nX+/PNPjh49SuvWrZXPHqzab2dnR0bGv8tlc6uOv2XLFu7cuYOn\npyceHh5KEr7vwZqRAEuWLCEsLIywsDB27txJtWrVWLNmDWXKlGHTpk2EhISQnp7+GL+SEELkP7mH\n9oTuj8b8/PwYPXq0UrD4vqSkJMqXz6pZt2HDBuX4pUuXqFKlCgMGDMDDw4O//vorx/idO3fm/fff\np1WrVmZJ7EmEh4ezevVqpUp/SEiI2X20B7Vo0cLsHtz9LXCSkpKUgsRhYWGygEQIYXPkHtoTul+N\nvnz58vTv3z/b58OHD2fevHn4+vqa7Sq9ZcsWunbtSvfu3Tl79qzZdOKDunbtyl9//WXx6sacKvhD\n1mKOK1euUK9ePeVY5cqVKV68uNnWMfeNHDmS9PR0vL298fb2ZvHixUDWLgMbNmyge/fuXLhw4ZF7\npQkhREHQ6hwsftkqqbZvowz71+fdKA8ae3WjU2uwlUohWpWVQjJvXlXdB6MVdlDYrLZSyKnsq3gf\nm9EKMww2UCnEeFd9lY+MlFTVMZIuxquOUfmjL1XHuDh5iMVtn5/9terr5QepFCKEEMKm741ZShKa\nEEIISWhCCCEKB1te7GEpSWg2yv65F1THyHR5Lu9Gj6BNva26D9a4/zW6kboanwBv+ddSdf5zTWqo\n7sOhRbtVx5jQ3VnV+d7DaqvuQ0b8xbwb5eHOyT9Vx8hUWaW+TDv1ew9c+zFYdQwHp6KqY1iDjNCE\nEEIUCoUhoRXYN0hISGDcuHG0b9+eHj160Lt3b3bs2FFQ3VHcvHkTDw8PEhISlGOBgYGsXLkyz3Nr\n1aql1Jl8sGZjXFwc3t7eAPz555/s2bMnfzovhBBPyM7B3uKXrSqwhDZq1CiaNWvG9u3bCQkJYeHC\nhcTHW758Nb8eTnZ1deXNN9/kk08+AeDEiRNER0czbNiwPK/v6OhIaGgoGzdu5L333mP+/PnZ2pw8\neZK9e/fmS9+FEOJJFYZKIQWSag8cOICDgwM9e/ZUjlWoUIF+/foBWSOaiRMncu/ePQACAgJo0KAB\nUVFRLF68GBcXF86fP8/WrVsZNWoUV69exWAwMHDgQPz9/QEIDg5m1apVlChRgpdffpkiRYowZcoU\nbt68yfTp07ly5QqQVTG/UaNGZv3r2bMnoaGhHDx4kIULFxIQEICdnR2hoaFs27aNlJQUjEZjtsr8\nD9ejLFGihNnnGRkZLFmyhLS0NI4cOcIbb7xB586drfSrCiHEk7PlRGWpAkloZ8+epXbt3G9O369O\nr9PpiI2N5b333iMkJATIGuFERERQsWJFAGbPno2LiwtpaWn4+fnRoUMH0tLSWL58ORs3bqRYsWIM\nHDiQWrWyFgXMnDmTwYMH06hRI65cucKwYcPYvHmz2fU1Gg3Tp09n0KBBeHp60rhxY+WzU6dOsWnT\nJooXL56t32lpaej1elJTU7lx4wZr1641+9ze3p4xY8Zw4sQJpkyZ8mQ/nhBC5ANZ5WglgYGBREdH\no9PpCA4OJj09Pdfq9PXq1VOSGcDatWuVe29Xr14lNjaWa9eu0axZMyXpdOrUSYlx4MAB/v77b2U0\nlZKSwr1797KVo3Jzc6NmzZr06dPH7Pirr76aYzIDKFq0KKGhoQAcPXqUiRMnZttuRgghbJHWzro7\nUe/du5dZs2ZhMpno0aMHb7zxRo7tYmJi6NOnDwsXLqRDhw6qrlkgCa1GjRps27ZNeR8QEMCtW7fw\n8/MDMKtOn5mZabZX2oOJJyoqit9//53g4GB0Oh0DBgxQKvPnVtHLZDKxfv16HBwsK72jfehvLQ9X\n189NgwYNuHXrFjdv3rSovRBCFCRrTjkajUZmzJjBmjVrKFeuHH5+fnh6elK9evVs7ebPn0+LFi2s\nct0CGWM2b94cg8FAUFCQcuz+/TKwvDp9UlISLi4u6HQ6zp07x7FjxwCoW7cuhw4dIikpiYyMDLPk\n+dprr7Fu3Trl/Z9/qn8e5r4Hk+i5c+cwGo2UKlXKrI2TkxPJyclWu6YQQliDNReFxMTEULVqVSpV\nqoSDgwNeXl5ERkZma/fNN9/QsWNHXF3V1Vq9r8AmTZcuXUpUVBTt2rWjZ8+eTJ48mQkTJgCWV6dv\n2bIlGRkZeHl5sXDhQho0aABkVfEfMWIEfn5+9OvXj8qVK+PsnPVA6ocffsjx48fx8fGha9euZkn1\nYblV4c+NwWBAr9fTvXt3xo0bx5w5c7LFaNasGWfPnkWv1ys7awshREHTOthb/MpLfHw8FSpUUN6X\nL1+ea9euZWuzY8cO+vbta7XvUGD30MqUKcOCBQty/Kxq1ar8/PPPyvtx48YB0LRpU5o2baoc1+l0\nfPllzlWmvby88Pf3JzMzk1GjRuHp6QlAqVKlWLhwoUV9fHAkB6DX69Hr9bm2P3HiRI7HK1WqxKZN\nmwAoUaIEP/30k0XXF0KIp+Vpr3KcNWuWMoiB3G8TPQ6bWBSSH5YsWcKBAwcwGAy89tprtGvXrqC7\nJIQQNsuaqxzLly/P5cuXlffx8fHKbaT7jh8/ztixYzGZTNy6dYu9e/dib2+vDD6eRKFNaJMmTSro\nLgghxDNDo7XeKse6dety8eJF4uLiKFu2LBEREdlm5B68pzZ58mTatm2rKplBIU5oQgghHoMVE5qd\nnR1Tp05l6NChmEwm/Pz8qF69OkFBQWg0Gnr16mW1az1IEpqNMulyXgjzODSGu+rOT1N3PoBG5U7R\noL5SPsDy4FOqzv+4xf9U9+GFts+rjlGlZhl1AezV77CsKeaiOoZz1UqqY6TfUbfjtPF2Qt6N8pCZ\nmqY6hs1U6LDyg9WtWrWiVatWZsd69+6dY9vZs2db5ZqS0IQQQqBx0BV0F1R7opTct29fswK7W7Zs\n4fXXX7dapwAWLVpEq1at0Ov1eHt7P/VK/MHBwcyaNUt5Hx8fb/YdV69eTYMGDUhJSck1Rt++fXN8\nzu3h2EIIUeC0dpa/bNQTJbSPPvqITz75BIPBwN27d1m0aBHTpk1T1ZGcHp4ePnw4oaGhLF68uEBq\nHz74DNnevXvNhs8RERG4u7s/caJ93GfchBAiP2m0WotftuqJphxfeuklPDw8WLlyJffu3aN79+5U\nrlyZsLAwvvvuOzIyMmjYsCEBAQFAVmmrkydPkpaWRufOnRk5ciQArVu3xsfHh/379/Pmm2/SsWPO\nO8hWq1YNgNu3b1OiRAliY2MJDAwkMTGRYsWK8fHHH1O1alUmTJhA8eLFOX78OLdu3WLWrFkEBwfz\nxx9/0LhxYz7++GMANm7cyKpVqwDw8PBg7NixgHmF/po1a+Lk5KT04ddff1Weh7tw4QJ2dna8/vrr\nfPHFF/j4+ACQmprKpEmTOHv2LNWrV8dgMCjnPyq2EEIUOBseeVnqie+hjRo1Cr1ej06nIyQkhDNn\nzrB9+3Z+/PFHtFotAQEBRERE4OXlxfjx43FxcSEzM5OBAwfSsWNHpaZX6dKl2bBhwyOvFRMTQ9Gi\nRZXtWKZOncrMmTOpUqUKR44cITAwkNWrVwOQnJzM+vXr2bZtGyNGjCA4OJgXX3wRvV7P2bNnKV68\nOIsXLyY0NBRnZ2cGDx7Mnj17cHNzY/ny5YSFhVGsWDH69eunVB7JzMzkn3/+oWrVqgCEh4fTuXNn\nGjduzLlz50hMTKRkyZJ8++23lChRgoiICE6dOqXUpoyPj881thBC2IT/ckJzdHSkS5cuODk54eDg\nwG+//cbx48fp0aMHJpOJtLQ0pfTJpk2bCAkJISMjg+vXr3Pu3DkloXXp0iXXa6xatYoff/yR2NhY\nli5dCmTVbzx27BhjxoxRniw3Go3KOW3btgWgZs2aPPfcc8rornr16sTFxZGWlkbz5s2V5Ni1a1cO\nHTpEamoqzZs3x8UlawVX586dlT3T/u///o+GDRsq19i8ebPZCO+XX36hV69eHD58mOHDhwNZu1fX\nqFEDyKq8n1tsIYSwBbY8lWgpVasctVqtUo3+/hYBY8aMMWsTGxvLunXrCAkJwdnZmQkTJigV8eHR\n1euHDx/OwIED2bFjB5999hmtW7fGZDLh6uqqbNPyMJ1Op/Tt/j/ff5+RkaH09WEajSbX0it79+6l\nZcuWQNZ+aJcuXWLAgAGYTCbS09N54YUXcnyu4sF41ijrIoQQ+aYQjNCslpJfffVVtmzZwq1btwBI\nTEzkypX0ZUohAAAgAElEQVQrJCcn4+zsjJOTE9euXWPfvn2PHbtdu3aULVuWLVu24OLiQtmyZZXF\nGCaTKdeK+Tklkfr16xMVFcXt27fJyMggIiKCV155hfr163Pw4EHu3LmDwWBg69atyjkHDx7E3d0d\nyJpuHDt2LJGRkezcuZNff/2VuLg44uPjadKkiVKz8c8//+Ts2bNA1lYyucUWQghboHHQWfyyVVZ7\nDq1mzZqMHj2aIUOGYDQacXBw4KOPPqJOnTpUr16dzp07U6lSJbPdnx9npd/IkSOZMmUKnTt3Zv78\n+UyfPp0lS5aQkZGBj48Pbm5u2eI9+P7+P5cvX5533nmH/v37A1lThq1btwbgrbfewt/fn5IlS1Kz\nZk0Abty4gbOzM0WLFgWyHlF4eCdqT09PtmzZwoABA5g4cSJeXl7UqFGD//3vf8o1c4othBA2oxBM\nOWpMMhf2SKGhoSQmJjJkyJCnet3MS3+ojmHSqvv7ivbebdV9QKP+f5I/Jqt7JASsUClkcQ/Vfbh6\nUP3ee2+/oq5G6fa26iuFZN66rj7G9TjVMdRWCilatXrejfJwKUz9FlD2TuqrAr0wZ23ejfJgOBBi\ncVtdc/X/P+QHqRSSh0dtFyOEEIWFNYsTFxRJaEIIIQrFlKMkNCGEEDJCE/kn06m06hh2l9XdN/rn\nu29V96GCr/op2+ea1FAdQ221/CnvWH5/ITeBn3ZTHcP9JXXV9g2nw1X3QVtC/X+bmfdyr4FqqdQE\ndfd4z28OUt2H0rUqq46RkWrIu9HTIAlNCCFEYaBxcCjoLqj27E+aPqRWrVro9Xq6d++OXq832wY8\nJx4eHiQmJgKYVQPJKWa3bt3w9fXl6NGjj4yZlJTE999/r7yPiopixIgRj/lNhBDiKSoE1fYL3QjN\n0dEx1yoiOcnpWbVHxdy3bx/z58/nm2++yTXm7du3+eGHH+jbt6/F/RBCiAJlw4nKUoVuhJbTY3Wh\noaHMmDFDeT9ixAgOHTqUa/tHxUxKSlLqQKakpDB48GB8fX3x8fFh586dACxYsIBLly6h1+v59NNP\nAbh79y5jxoyhc+fOTJgw4cm/oBBC5IP/7PYxtiwtLQ29Xo/JZKJKlSosWbIEULf/2P2Yqamp3Lhx\nQ6kUUrRoUZYuXYqTkxO3bt2iV69eeHh4MG7cOM6cOaOM6qKiovjzzz+JiIigbNmy9OnThyNHjtCo\nUSP1X1gIIayhEIzQCl1CK1q06GNNOT5uzKNHjzJx4kTCw8MxGo0sWLCAQ4cOodVquXbtGgkJCTnG\nqFevHuXKlQPAzc2NuLg4SWhCCNthhao+Ba3QJbSc2NnZmW0x82C1/8fVoEEDbt26xc2bN9mzZw+3\nbt0iLCwMrVaLh4dHrrEdHlhBZGdnl+MO3UIIUWAKQUJ79r/BQ3K6J1apUiVOnTqFyWTiypUrxMTE\nWHzuw8fPnTuH0WikVKlSJCUl4erqilar5ffff1dWVDo5OXH37l0rfBshhHg6TFp7i1+2ynZ79oRy\nulfWuHFjKlWqhJeXF9WrV6d27do5ts/tPpvBYFDuywHMmTMHjUaDt7c3b731Fj4+PsquAgAlS5ak\nYcOGeHt706pVK6Waf17XEUKIAlMI/lwqdAntyJEjOR6fN29ejscjIyPzPPfEiRM5Hi9VqhRBQTlX\nG5g/f77Z+6ZNmyr/PGXKlBzPEUKIAmPDqxctVegSmhBCiMdnKgT30CShCSGEKBSLQiShCSGEKBQJ\nTXastlGPs3tsbuwqqNuR11i0uOo+GE/uVx1je/9PVcd4oe3zqs4v11B9xf+ACRtVx/iqgYeq85Pm\nt1HdB8M/f6uOkXI15+c1H8e9BHU7Vlfu2091H84s+lx1DJ1LMdUxqi9Wv3NAxpUzFre1r/CS6uvl\nBxmhCSGEsOnl+JZ65seYffv2Ze/evcr7LVu28Prrr1v1GosWLaJVq1bo9Xq8vLzM6kLmJiYmhk8+\n+QSA4OBgZs+eDcD27ds5f/68VfsnhBCqaTSWv2zUM5+SP/roI9555x3c3d1JT09n0aJFrF69WlXM\nzMxM7OzM65oNHz6cgQMHYjKZ6N27N9HR0TRu3DjXGPXq1aNevXrZjm/btg2NRsOLL76oqo9CCGFV\nheAe2jP/DV566SU8PDxYuXIly5Yto3v37lSuXJmwsDD8/f3R6/UEBgYq7QMCAvDz88Pb25tly5Yp\nx1u3bs38+fPx9fVlx44d2a5z/1ZjWloa6enpuLi4AFkjxD///BOAGzdu0KFDBwAOHDjAqFGjzGIc\nPnyYvXv38sknn1i0V5sQQjwtJo3W4peteuZHaACjRo1Cr9ej0+kICQnhzJkzbN++nR9//BGtVktA\nQAARERF4eXkxfvx4XFxcyMzMZODAgXTs2FGp8FG6dGk2bNiQ4zVWr15NWFgYly9fpm3btrz0Us43\nRR9VeaRJkya0atWKTp064enpaaVvL4QQViAPVtsGR0dHunTpgpOTEw4ODvz2228cP36cHj16YDKZ\nSEtLo0KFCgBs2rSJkJAQMjIyuH79OufOnVMSWpcuXXK9xv0px4yMDEaNGsW2bduU0ZgQQjzzbHjk\nZalCkdAAtFot2v//NwyTyUSPHj0YM2aMWZvY2FjWrVtHSEgIzs7OTJgwwaw6frFieS+ftbe3p2XL\nlhw6dIgOHTpgZ2dnNh0phBDPJFnlaJteffVVtmzZwq1btwBITEzkypUrJCcn4+zsjJOTE9euXWPf\nvn0Wx7yftEwmE//3f//H889nPddUuXJljh8/DsDWrVvzjOPk5ERycvLjfiUhhMhXheEemu32TIWa\nNWsyevRohgwZgo+PD8OGDSMhIYHatWtTvXp1OnfuzOTJk81WKeZVAf+rr75Cr9fj4+ODnZ0dvXr1\nAmDYsGGsW7cOX19fkpKS8uxb165d+eKLL2RRiBDCtmi0lr8ssHfvXjp16kTHjh1ZuXJljm0+/vhj\nOnToQLdu3Th16pT6ryCVQmyTVAr5l1QK+ZdUCvmXVAr5lzUqhaQl37a4bRHnEo/83Gg00rFjR9as\nWUO5cuXw8/NjwYIFynoFgD179vDdd9+xcuVKjh07xsyZM1m/fv0T9x8K6QhNCCHEY7LiCC0mJoaq\nVatSqVIlHBwc8PLyMtuqC7K27urevTsA9evXJykpiRs3bqj6CpLQhBBCWPUeWnx8vLKyHKB8+fJc\nu3bNrM21a9d47rnnzNrEx8er+g7P/rIWIYQQ6tnwYg9LSUKzUbtcW6qOUdWxqKrzizmo/w88ulx7\n1TEmdHdWHaNKzTKqznd/Sd35AF81yHvRUF6GHt2p6vzN5Waq7sN5hxTVMf4pcU91jOtJqarOf6uE\n+vJzAY0nqo5RopiD6hjq77iDEevVaCxfvrzZorf4+HjKlStn1qZcuXJcvXpVeX/16lXKly+v6rrP\nfkoWQgihmtFksviVl7p163Lx4kXi4uIwGAxERERkq47k6elJWFgYAEePHsXFxYUyZdT9xTHPhPY0\nqtnn5MCBAzRp0gS9Xo+3tzfDhg1Tnit7lHfeeYdu3brx7bffWrU/KSkpvPfee3h7e+Pt7U3//v3z\nfJC6devW8syZEOKZYHqMV17s7OyYOnUqQ4cOpWvXrnh5eVG9enWCgoL48ccfgaw/HytXrkz79u0J\nCAhg2rRpqr9DnlOOT6uafU7c3d35/POsZbFz584lKCiIt956K9f2V69e5fTp02zZssXqfVmzZg0V\nK1ZkwYIFAJw/fx57+0f/fHk92yaEELbCaOUHuFq1akWrVq3MjvXu3dvsfUBAgFWvmWdCe7Ca/b17\n98yq2X/33XdkZGTQsGFDpWMBAQGcPHmStLQ0OnfuzMiRI4GsbOzj48P+/ft58803uXLlCsHBwdjb\n2/Pyyy8zd+7cbNd+sDpHSkoKlSpVArJGSzNmzODcuXNkZGQwZswY2rRpw7Bhw7hy5Qp6vZ5p06ah\n0+mYPn06aWlpVK1alVmzZuHs7Ezfvn2pW7cu0dHR+Pj44OXlxfTp07ly5QparZYpU6Zk2/rl+vXr\nVKtWTXn/4PYvI0aM4Pr16xgMBgYOHIi/v79Z/4Fcfy8hhLAFheGRZIsWheRHNfsWLVqwe/du7O3t\nc52WO3jwIHq9nps3b1K8eHEmTZoEwNKlS2nZsiWzZ8/mzp07+Pv78+qrr7J8+XLeeecdQkNDAfDy\n8mLmzJk0aNCAhQsXsmzZMiZOzLqJazQa+emnnwAYO3Ysr7/+OvXq1SMuLo4RI0awadMms774+fkx\nfPhwNm/ejLu7O3q9Xil/NXfuXFxcXEhNTaVHjx506tSJ4sX/fSj5Ub+XEELYAmuP0AqCRQktP6rZ\n16xZk/Hjx+Pp6Um7du1yvO6DU44rVqxg3rx5TJ06lf379/Prr78q5VTS09O5cuWK2RRfYmIiBoOB\nBg0aANC9e3clIT7cl99++40LFy4of0NJSkrCYDCg0+mUNrVr1yYyMpL9+/ezf/9+/Pz8CA4OpmrV\nqnz11Vfs2rULyFrNc/HiRWrXrq3051G/lxBC2IJCkM8sX7Zv7Wr2q1evJioqisjISL744gs2bdr0\nyHtOHh4eTJgwQbn+0qVLqVKlilmbixcvmr1/1BDa0dHR7P1PP/2U5720YsWK0b59e9q3b4/JZGLv\n3r3UqFGD6OhogoOD0el09O3bN9tikdx+LyGEsBWZhWCI9kTL9tVWszcajVy5coVmzZoxfvx4EhMT\nuXcv+3MpDyakw4cPK1N8LVq04JtvvlE+e7Co5f1zSpYsiaOjI0ePHgVg48aNvPLKK7l+nwfj3d+B\n+kHR0dFK8WGDwcDff/9NxYoVSUpKomTJkuh0Os6cOcMff/yRrS+5/V5CCGErjI/xslVP9GD1g9Xs\njUYjDg4OfPTRR9SpU0epZl+pUqVcq9lnZGQwbtw4UlJSMBqNDBs2LMe9yKKiotDr9RiNRlxcXJg1\naxYAo0ePZtasWXh7ewPw/PPPs3Tp0mzXmTt3LtOmTTNbFPJwG4CpU6cyffp0NmzYgNFopFmzZkyd\nOtWsTWxsLNOnT0ej0WAymfDw8MDT0xODwcCPP/5I165defHFF5Upzgevk9vvJdOOQghbUQjWhEi1\nfVv1y1/X8m6Uh6olbaBSyGX11TEmLDugOoYtVAqZP2We6hhqK4V0/EP9b3n+lhUqhdy0gUohLaxQ\nKSRc/ZYnVqkUMrSZ6hhxt+5a3LZSKSfV18sPUvpKCCHEf2fZvhBCiMLNlu+NWUoSmhBCCLmHJvLP\nreXvq45h76TuHpqdo/qddK1BV622+iD26u5TGE4fVd0Fu9LP5d0oD5tV7l7wS93mqvvQtYL6ncwb\nv90q70Z5KFpSXT9unb6Yd6M8OFUorTrG1cNnVceov97ycn+5+fuG5fe7q5VR/99AfpARmhBCiP9O\npZBnwQcffMDu3bspXbp0trJVD4uKisLBwYGGDRtm+yw0NJS5c+fy3HPPYTKZcHNz45NPPmHy5Mm0\nbduWDh06PDL2+fPnCQgIICkpifT0dBo3bkxgYCBRUVGMHDmSKlWqYDKZcHV15auvvlL1nYUQwloK\nw1xdoUlovr6+DBgwQKnV+ChRUVEUK1Ysx4QGWTUgp0yZYtF1jUajUkEF4OOPP2bo0KG0bdsWyKrj\neF+TJk1YsWKFRXGFEOJpMhaC4leFZoPPJk2a4OLiku34unXr8PLyolu3bowbN464uDiCgoJYu3Yt\ner2e6OjobOfkdVvRw8ODefPm4evry9atW80+u3Hjhtmuqy+99NITfiMhhHh6TCbLX7aq0IzQcvPl\nl1+yc+dOHBwclNJcvXv3xsnJiSFDhuR4zubNmzly5AgAAwcORK/XZ2tTqlQpZeeABw0aNIiBAwfS\nqFEjXn31VXr06KFU3j98+LASq1OnTrz55pvW+ppCCKGK3EN7Bri5uTFu3DjatWuXa1X/h1ky5fhg\ntf4H+fr60rJlS3799Vd27NjB+vXr2bhxIyBTjkII22XLIy9LFZopx9ysXLmS/v37c/LkSfz8/DAa\nrfP44MPV+h9UtmxZfH19WbZsGXZ2dmb30YQQwhZlmkwWv2xVoUpoD9/7MplMXL58maZNmzJu3DiS\nk5NJSUnByckp101F1fr111/JyMgAsna5TkxMNLunJoQQtshoMln8slWFZspx3LhxHDx4kMTERNq0\nacPbb79Nt27dmDBhAsnJyZhMJgYOHIizszNt27ZlzJgx7Ny5kylTppjtCmCJR+3btn//fmbOnEnR\nolkPNU+aNInSpUtz7tw5Vd9PCCHyU2YhqH0llUJslFQK+ZdUCvmXVAr5l1QK+Zc1KoUcunjL4rav\nPF9K9fXyQ6EZoQkhhHhytnxvzFKS0IQQQtj0vTFLSUKzUUkX41XHKNda3fSSXalyqvuQfkH9BogZ\n8eqnhjTFsj90/zi0JdRPLRn++Vt1jPMO6jbXtMZ0YfgV9Zu2vmJX8OvRXGu9oDrGzVMXVMcwGjJV\nx7CG9ExJaEIIIQoBmXIUQghRKEilkMeQmJjI4MGD0Wg0XL9+Ha1Wi6urKxqNhuDgYOztzbty+/Zt\ntmzZQu/evR8ZNzMzE3d3dw4dOpTteJ06dXBzc8NkMmFnZ8e0adOoV6+equ9x8uRJEhISaNmyZbbP\nUlJSmDJlivIgdYkSJVi9ejX29vZmfdFoNKxYsUKeTxNC2IzMQpDRnlpCK1myJGFhYQB8/vnnj6yl\nCFkJMCgoKM+EBrk/F1a8eHFCQ0MB2LNnDwsWLGDNmjWP3/kHnDhxgjNnzuSY0NasWUPFihVZsGAB\nkLWVzP1E/WBfhBDC1hSGRSEFf2eWrALC3t7eeHt789133wEwf/58Lly4gF6vZ8GCBSQnJzNo0CB8\nfX3p1q0bu3fvzjPug4/YJSUlUaJECQDi4+Pp27cver0eb29vjh49SmZmJq+88gqffPIJXbt2Zdiw\nYRw9epT+/fvTvn179u7dS1paGsuWLSM8PBy9Xs+2bdvMrnf9+nWzUdeLL76InZ1dtr4IIYStyTRZ\n/rJVBX4PLSYmhoiICEJCQkhPT8ff35+mTZsyfvx4Ll26pIxqMjMzWbZsGU5OTty8eZM+ffrQpk2b\nR8a+e/cuer2e1NRUEhISWLduHQA///wzHh4eDB8+HJPJRGpqKpCV9Fq3bs3777/PiBEjWLp0KevW\nrePUqVMEBAQQEhLCyJEjOXv2LJMnT852PT8/P4YPH87mzZtxd3dHr9fz/PPPm/XFZDJRtWpVFi9e\nbMVfUQgh1CkMI7QCT2jR0dF06NABnU6HTqejXbt2HD58mNdee82sndFoZN68eURHR6PVarl69SqJ\niYnK1iw5cXZ2VhJidHQ0EyZMYNOmTdStW5dp06aRlpaGp6cnbm5uZGZm4ujoSPPmWUvdX375ZYoX\nL45Wq6VmzZpcvnw5z+9Su3ZtIiMj2b9/P/v378fPz4/g4GAqV65s1hchhLA16YWg9pVNTDlaIiws\njOTkZDZu3EhYWBglS5YkLS3N4vMbN27MtWvXuHPnDu7u7nzzzTeUK1eOSZMmER4eDoCDw7/lkTQa\nDTqdDgCtVktmpmXPihQrVoz27dszffp0OnfuzN69ex/jWwohRMEoDFOOBZ7QGjduzI4dOzAYDNy9\ne5fIyEiaNGmCk5MTd+/eVdolJydTunRpNBoN+/fvJz7+3wePc7s/9eDxM2fOYGdnh4uLC5cvX6ZM\nmTL4+/vj6+vLyZMnHxnnwc8eVak/OjqapKSsh04NBgN///03lSpVyjO2EEIUNKm2bwX16tXDy8uL\nHj16ANC3b19eeuklIGsKz8fHhzZt2jB48GBGjBiBj48P9erV44UXXlBi5LbKMSUlRblvpdFomDt3\nLgAHDhxgzZo12Nvb4+zsrBx/VBX9+5+5u7uzevVqfH19GTFiBB06dFDaxMbGMn36dDQaDSaTibZt\n2+Lh4UFmZuYjYwshREEzFoJl+1Jt30ZdnJz7Iw2WKiylr7TFS6qOobb0lcmQqroP6VcvqY7x5YsD\nVJ1fs1d31X2wRumrj+Z4q46hK+6k6nyNFcpvWaP01e3zN1THaBi2Le9GefjmyD8Wtx3QqLLq6+WH\nAp9yFEIIUfCe1pTj7du3GTp0KB07dmTYsGHKbZoc+2Q0otfrGTFihEWxJaEJIYQg02Sy+KXGypUr\nad68Ob/88gvNmjXjiy++yLXtunXrqF69usWxC/wemshZ+fZtVcfQOOhUnZ95O0F1H9RurAlw5+Sf\nqmM4V62k6vzMe+qq3AOkXFX/e/5T4p6q8/tYYWNNa1TKnzZpk+oYtYoXUXV+7ykd8m6UB5Px2V/q\nft/TqrYfGRnJt99+C4Ber2fAgAGMHz8+W7urV6+yZ88eRowYwddff21RbEloQgghntrqxZs3b1Km\nTBkAypYty82bN3NsN2vWLCZOnPjIKcmHSUITQghh1e1jhgwZwo0b2Re7vPvuu9mO5bQCfPfu3ZQp\nU4ZatWpx8OBBi69rEwktISGBWbNmERMTg4uLCw4ODgwfPpx27doVdNcUM2fO5JdffpEHpYUQhZI1\nq+0/aoqwdOnS3LhxgzJlynD9+nVcXV2ztTly5Ag7d+5kz549pKWlcffuXSZOnKg8YpUbm1gUMmrU\nKJo1a8b27dsJCQlh4cKFZg9O58XSKh5PymQysWvXLmrUqEFUVFSB9EEIIfJTptFk8UsNDw8PNmzY\nAEBoaCienp7Z2rz33nvs3r2byMhIFixYQLNmzfJMZmADI7QDBw7g4OBAz549lWMVKlSgX79+AMTF\nxTFx4kTu3cu6GR4QEECDBg2Iiopi8eLFuLi4cP78ebZu3cqoUaO4evUqBoOBgQMH4u/vD0BwcDCr\nVq2iRIkSvPzyyxQpUoQpU6Zw8+ZNpk+fzpUrVwCYPHkyjRo1ytbHgwcP4ubmRpcuXQgPD6dp06ZA\n1jY4Fy9e5NKlS1SsWJFPP/2UefPmcejQIQwGA/369aNnz56kpKQwcuRI7ty5Q0ZGBu+8806O/xKF\nEKKgPK390F5//XXeffddQkJCqFSpEosWLQLg2rVrTJ069ZGrHvNS4Ant7Nmz1K5dO9fPy5Qpw9df\nf41OpyM2Npb33nuPkJAQIGuzzYiICCpWrAjA7NmzcXFxIS0tDT8/Pzp06EBaWhrLly9n48aNFCtW\njIEDB1KrVi0gaxpx8ODBNGrUiCtXrjBs2DA2b96crQ8RERF06dKFNm3aMHfuXDIzM5VtYc6dO8cP\nP/yATqdj/fr1uLi4EBwcjMFgoE+fPrz22mtUqFCBpUuX4uTkxK1bt+jVq5ckNCGETXlaCa1kyZI5\n7ktZrly5HJNZ06ZNlUFEXgo8oT0sMDCQ6OhodDodwcHBpKenExgYyKlTp7CzsyM2NlZpW69ePSWZ\nAaxdu5YdO3YAWUs+Y2NjuXbtGs2aNVOq8nfq1EmJceDAAf7++2+lzmJKSgr37t3D0dFRiZmens6e\nPXv44IMPcHR0pF69euzbt4/WrVsDWcPn+0WM9+3bx+nTp9m6dSuQVX8yNjaW8uXLM3/+fA4fPoxW\nq+XatWskJCRQunTp/PoZhRDisRgynv1HEAo8odWoUcNso8yAgABu3bqFn58fkLULdJkyZdi0aROZ\nmZnUr19faftg4omKiuL3338nODgYnU7HgAEDlGr8jypevH79erMq+w/bt28fSUlJdO3aFZPJRFpa\nGkWKFFESWrFixczaT506NdvWN6GhoSQmJhIWFoZWq8XDw+OxdgoQQoj89rRGaPmpwBeFNG/eHIPB\nQFBQkHLs/v0yyNp0s1y5rJqCYWFhuS6+SEpKwsXFBZ1Ox7lz5zh27BgAdevW5dChQyQlJZGRkWGW\nPF977TVl00+AP//M/gBveHg4M2fOJDIykp07d7Jjxw5+++23HBNSixYt+P7778nIyADgwoUL3Lt3\nj6SkJFxdXdFqtfz+++8W7a0mhBBP09NaFJKfCnyEBrB06VJmzZrFqlWrcHV1xdHRkQkTJgBZ1fff\nfvttwsLCaNmypdmo7EEtW7YkKCgILy8vXnzxRRo0aABA+fLlGTFiBH5+fpQsWZJq1arh7OwMwIcf\nfkhgYCA+Pj4YjUaaNGnC9OnTlZipqans27ePwMBA5ZijoyONGzdm586d2frg7+9PXFwcer0eAFdX\nV5YuXYq3tzdvvfUWPj4+1KlT57FKuQghxNNgy4nKUv+JavspKSkUK1aMzMxMRo0ahZ+fn00945aT\ntJ3r8m6UB7Wlr4z37ubdKK8YSbdUx0g+r75KfWEpfTXn5TdVnT/h5HLVfdBK6StFctx11THuxOZc\nKeNxWKPa/sRNJyxuO9c794V8BckmRmj5bcmSJRw4cACDwcBrr71m88lMCCGetoxCMEL7TyS0SZMm\nFXQXhBDCphWGKcf/REJ7FtmVUL+kPyPhqqrzTanqpxzVTnsCZKZnqI6RfueOqvNTE26r7sO9BHV9\nALiepG6j0aIli6vugzWonS4EOJWkbqWwMVP9MnXDHfVT0bbCYIXfo6BJQhNCCCEjNCGEEIWDJDQr\nsvWK+zExMcydO5eEhAQcHR2pXbs2U6ZMoUgR9VMnQghR0CShWdGoUaPw9fVl/vz5AFy5ciXHZ71y\n82B9RWtLSEjg3XffZdGiRdSrVw+Abdu2cffuXYsSWn72TQghrCGzEOy+bRMJzdYr7n/33Xfo9Xol\nmQF06JD1DEtMTAyzZs3CYDBQpEgRZs+ezQsvvEBoaCjbtm0jJSUFo9HIggULGDt2LHfv3iUjI4Pp\n06fTuHHj/PtRhRDiMcgIzUpsveL+mTNnlOofD6tevTrff/89Wq2WAwcOsGDBAj777DMATp06xaZN\nmyhevDhff/01LVu25M0338RkMpmV9xJCiIKWJsWJ84etVdx/lKSkJCZNmqTEe7DW5Kuvvqpcs27d\nunz44Yekp6fTrl073NzcnvTnEUIIq5MRmpXYesX9GjVqcPz4cTw8PLJ9tnjxYtzd3fn888+Ji4tj\n4BJGBBUAACAASURBVMCBymcPVuJv0qQJ3377Lbt37+b9999nyJAhdOvWLa+fRgghnorCkNAKvNo+\n2H7F/f79+xMWFkZMTIxybPv27SQkJJCcnEz58uUBlG3Fc3L58mVKly6Nv78//v7+nDx58pG/iRBC\nPE1Sbd+KbLXiPkDp0qVZuHAhc+bM4ebNm2i1Wpo0aUKrVq0YNmwYkyZNYvny5coeaTmJiopi9erV\n2Nvb4+TkxJw5c6zwqwkhhHXYcqKy1H+i2j48exX3M6Ij1MewgdJX1pB44i/VMYqVK6XqfFspfTW1\n7tuqzv8s6SfVfbCGdZNCVcdQW/pq2iwv1X1IPK1+J4jUW+rKmYF1qu23+3yfxW13jG6h+nr5wWZG\naPlNKu4LIUTuTIVghPafSWhScV8IIXKXKcWJRX7RliyrOkbyfssrreTk11lbVfehw8I+qmOUaddR\ndQzjbXWba57fHJR3ozzUCRinOsZbJV5Udf6txRdV98G11guqY1hjc0211fI/+kD9tP74UU1Vx7h+\n8obqGNYgIzQhhBCFglESmhBCiMLA9OzPONrGc2h5SUhIYNy4cbRv354ePXrQu3dvpRpIXho2bJjt\nWFBQEBs3bnysPmRmZtK8eXMWLFjwWOcJIcSzwGQyWfyyVc/ECM3SSvw5VbXXaDTZ2vXu3fux+7B/\n/37+97//8csvv/Dee+/l2MZoNKLVPhN/RxBCCDOFYcrR5v/0zasSf2hoKG+99RaDBg1i8ODBFsX8\n/PPP+frrr/n777+VavyQVdXf29s7x3MiIiLo27cvVapU4ejRo8pxDw8P5s2bh6+vL1u3buXSpUsM\nHz6cHj160L9/f86fPw/Arl276NmzJ76+vgwdOpSbN28+7k8hhBD5xmQ0WfyyVTY/QsurEj+YV7V/\nHNWqVSMjI4O4uDgqVarE5s2b8fLK/rClwWDg4MGDzJgxg8TERMLDw5UqJAClSpVSyl4NHjyYwMBA\nnn/+eWJiYpg+fTpr166lSZMmrF+/HsjayubLL7+URwmEEDZDlu0XgIcr8YN5VfvH1alTJzZv3szr\nr7/Oli1bWLRoUbY2u3btomnTpuh0Ojw9PVmyZAkffvihMp3ZpUsXIKsayf/93//xzjvvKPPMGRkZ\nQNY06bvvvsu1a9fIyMigcuXKT9RfIYTID7Y88rKUzSe0vCrxg3lV+8fVpUsX3nnnHdq3b49Go+H5\n55/P1iYiIoIjR47g6emJyWTi9u3b/P777zRv3hz4t+K/0WjExcWF0NDsZX1mzJjBsGHDaNOmDVFR\nUXz++edP3GchhLC2wpDQbP4eWl6V+POS14qcKlWqoNVqWbZsmTLSelBycjKHDx9m9+7dREZGsnPn\nTgICAti0aVO2ts7OzlSuXJmtW/99IPl+9f67d+8qOwbklPCEEKIgGY0mi1+2yuZHaPDoSvx5SUtL\no02bNphMJjQaTY4LR7p06cKnn37Ku+++m+2zHTt20Lx5c+zt//2pPDw8+PTTTzEYDNlWUc6bN49p\n06axfPlyMjMz6dKlC25ubowaNYoxY8ZQokQJ3N3diYuLe7wfQQgh8pEtL8e31DOR0MqUKZPr8196\nvR69Xp/ruZbsOzZ06FCGDh2a42fdu3ene/fuZsdKlCjBb7/9BkBkZKTZZ5UqVWLVqlXZ4nh6euLp\n6ZlnX4QQoiAUhgern4mEJoQQIn/Z8lSipSShCSGEwJjx7A/R/jMbfD5rDAdCVMdIv3BK1fkae536\nPtxRv6nlzVOxqmNkpqrbDLJIySd7LORByXHXVcd4u/FEVed/l/Kj6j7cvaJu5wIAk1H9H56GOymq\nzndwKqq6D/OWRqmO0eU5Z9UxfK4cVx2j1tifLW57aqGP6uvlBxmhCSGEeGrL9m/fvs3YsWOJi4uj\ncuXKLFq0KMfniL/44gt+/vlntFotNWvWZPbs2eh0j/5Lts0s24+Pj2fkyJF07NiRDh06MGvWLOWh\n5Px0v3jxo8penTlzhkGDBtGpUyc6duzIsmXLVF3Tw8ODxMREVTGEEMKanlbpq5UrV9K8eXN++eUX\nmjVrxhdffJGtTVxcHOvXryc09P+1d95RTWVdG38CiCJYsOuMvWIbGyiioIyKNGliRWcc9bVhR8YG\nqNiwY0Fn1JEBC76AojQLKBYUC2LBEUcBAQEVATXUkOR8f/DlvkQQ7k0iGji/tVgruTdn5yQk2fec\nvfezzyI4OBgikQhhYWFV2v5uHJqjoyPGjBmDixcv4uLFi8jPz1eIsr1IJKr0fEXixWUpLi7G/Pnz\nMXfuXFy4cAHnz59HXFwcTpw4IfOcqnpOCoVCqW6qqw4tMjKSyUy3sbGpsHOKlpYW6tSpg8LCQgiF\nQhQVFTF1vJXxXTi027dvo169ekx6PI/Hw6pVq3DmzBkUFRVh4sSJSExMZB4/bdo0PH36FIWFhVi9\nejUj+itR4P9csLigoAC//vorbG1tMW7cuHKp9pURHByMgQMHMqogdevWhaurK5OaLxE6lmBpaYmM\njAwApV0C7OzsYGlpych0ATWj3oNCodQsqqt9TE5ODpo1awYAaN68eYVC7Y0aNcJvv/2GESNGwNDQ\nEA0aNMDQoUOrtP1dxNAqEiDW0tJCmzZtkJqaCjMzM4SFhWHhwoXIysrC+/fv0atXL+zevRv6+vrY\nvHkz+Hw+xo8fz7zosoLFYrEYBw4cgKamJnJzczFx4kTWNWEVza1t27YoKChAQUHlQektW7agYcOG\nKC4uxvjx4zFmzBg0atSIwztDoVAo1YMiY2gzZszA+/fvyx2vSLyioh2rtLQ0eHt74+rVq2jQoAEW\nLVqE4ODgL4aFJHwXDu1LSK4Exo4di5kzZ2LhwoUIDw+HiYkJAODmzZu4cuUKjh49CgAoKSlhVkdl\nBYvFYjF27dqFe/fuQUVFBe/evUN2djaaNm0q1/yqivH9/fffzHL6zZs3SElJQd++feV6TgqFQvka\niBSYs1B21+pzmjZtivfv36NZs2bIyspCkyZNyj3myZMnGDBgABo3bgwAGD16NOLi4pTDoXXp0gUX\nL16UOpaXl4fMzEy0b98edevWhba2Np4/f46wsDBs2LCBedy+ffvQoUMHqbGPHj2SEiwODg5Gbm4u\ngoKCoKKiAmNjYxQXs0vj7ty5M+7fvy91LC0tDfXq1UPDhg2hqqoKcZkUZIndu3fvIiYmBv7+/lBX\nV8e0adNYPyeFQqFUN0Rceb6BojA2NsaZM2fwn//8B2fPnq1wt6xTp044ePAgiouLoa6ujpiYGPTp\n06dK299FDE1fXx9FRUU4d+4cgNJEDg8PD9ja2qJu3boAAFNTUxw5cgT5+fno1q0bAGDYsGHw9fVl\n7Dx7VnHdFZ/PR5MmTaCiooKYmBhmFQdUHc8aN24cHjx4gNu3bwMAioqKsGnTJsyYMQNAqdSVRF7r\n6dOneP36NfOcDRs2hLq6OhITE/Ho0SPO7wuFQqFUF0QsYv0nD7Nnz8atW7dgYmKCmJgY/Oc//wEA\nvHv3DnPmzAEA9OjRA1ZWVkzeAyFEqsnzl/guVmhAqQCxm5sbvLy8QAiBoaEhli5dypw3MTHB5s2b\nMX/+fObY/PnzsWnTJlhaWoIQgh9//BGHDh0qZ9vS0hLz5s3DuHHj0Lt3b3Tu3Jk5V1XGYd26dXHw\n4EFs2LAB69atw7t37zBx4kRG5NjExATnzp2DpaUl+vbti44dOwIAhg8fDj8/P5ibm6Njx45SDUFp\nliOFQvneIFVkhCuKxo0bw9vbu9zxFi1aSKXwz5o1C7NmzeJkmyqFcCQyMhIHDhzAH3/8gebNm3+1\n56FKIf+DKoX8D6oU8j+oUsj/UIRSyI9Tj7J+7OsTM+V+vq/Bd7NCUxaoaj6FQqmJVFcM7WtCHRqF\nQqFQIBYKvvUU5IY6NAqFQqHQFRrl61HwJFZuG4JP+XKNFxbJf8UmLpG/tkURsQ6eqnwJvYp4L9Qb\n1q/6QVXQqH4duca/uf5S7jmIBcr/wwcAWf+UL/zliiLiX2Fv8uS2oQjtezF1aBQKhUKpCdAV2nfG\nwYMHERoaChUVFaiqqmL9+vVfVOZYtWoVRo4ciTFjxnzR3qpVq3D37l00bNgQPB4Pbm5u+Omnn8o9\nzs/PDxoaGrCyslLYa6FQKJTqhDq074iHDx/i2rVrCAoKgpqaGj58+ICSkhK57f7+++8YM2YMoqOj\n4eLigvPnpZvgiUQiTJo0Se7noVAolG9JddWhfU1qjEPLysqCtrY21NRKX5JEA+zAgQOIiopCUVER\n+vfvLyWbJeHp06fYunUrCgoKoK2tja1btzJq0BJ0dXWRlpYGoFTtX0dHBw8ePICFhQXy8vKgqamJ\nGTNmIDU1FW5ubsjJyYGqqio8PT3Rtm1bHD16FOHh4SgpKcHo0aPh6Oj4ld8RCoVCYU9NWKF9F9JX\nisDAwACZmZkYO3Ys1q9fj3v37gEodT7+/v4IDg5GUVERoqKipMYJhUK4u7tj7969CAwMhK2tbYV9\n2CIjIxnJLcm4gIAARjFEgpOTExwcHHDu3Dn4+fmhRYsWiI6ORkpKCgICAhAUFIT4+Phy+pAUCoXy\nLRELBaz/vldqzAqtfv36OHv2LO7fv4+YmBgsXboUTk5OqF+/Po4cOYLCwkJ8+vQJXbt2xYgRI5hx\nycnJePHiBX777TcQQiAWi6UayXl4eODgwYNo0qQJNm/ezBw3MzMrN4f8/Hy8e/eOKbyWtAu/efMm\noqOjYWNjA0IICgsLkZKSgkGDBn2ld4NCoVC4oQj1lm9NjXFoQKlGoq6uLnR1ddGtWzecPn0a//77\nLwIDA9GyZUvs37+/nOI9IQRdu3aFn59fhTYlMbTP0dDQqPDxX1ISmzNnDitxTQqFQvkW0C3H74jk\n5GSkpPxP8+/Zs2fo1KkTgNJ4Wn5+frkWNQDQsWNH5Obm4uHDhwBKtxJfvpStVkdTUxOtW7dmeqAJ\nBAIUFRVh2LBhCAwMZBqCvn37tsIurRQKhfKtqC61/a9JjVmhFRQUwN3dHXl5eVBVVUX79u2xYcMG\naGlpwcLCAs2bN6+wn06dOnXg6emJjRs3gs/nQywWY/r06ejSpcsXn6sytXwPDw+4urpi7969jG0D\nAwMkJSVh4sSJAEod3/bt2ytsbEehUCjfgppQWE3V9r9TPvy5Wm4bNUUpRBHI+1qISP74glgBZSTO\nPy2t+kGV4HphndxzqClKIfxM+RU6PqTK301CEUohh8gruW3U11/E+rEFt/fK/XxfgxqzQqNQKBSK\n7HzPW4lsoQ6NQqFQKN91Oj5bqEOjUCgUSo1YodEYGoVCoVBqBDUmbZ9CoVAotRvq0CgUCoVSI6AO\njUKhUCg1AurQKBQKhVIjoA6NQqFQKDUC6tAoFAqFUiOgDo1CoVAoNQLq0CisyMvLq/SvugkPD2d1\n7HsnIyOD1bHvlbdv3yrcZnZ2NjIyMpg/ZeHDhw+V/lG+PrSwWsmJjo6GgYEB68enpqaiVatWUFdX\nx507d/D8+XNYW1ujYcOGlY4zMjICj8cDIQTv3r2DpqYmCCEoKChAixYtcO3aNdZzePjwIdzd3ZGU\nlISSkhKIRCJoaGjgwYMHrG3Y2Njg7NmzVR77nM2bN1faLWHVqlWs5wAADx48QHp6OkSi/6ksWFtb\nsx4v6+v4nNDQUKSmpmLevHnIzMxEdnY2evfuXemY3377DX/99RcA4I8//sCcOXM4Pefnc128eDE8\nPT0525AQGRkJDw8PvHv3Dk2aNEFGRgY6d+6M0NBQ1jZiY2Oho6OD+vXr49y5c/jnn38wffp0/PDD\nD6zG5+Tk4PDhw3j58qVU70QfH58qxxobGzPfkc/h8XiIjIxk/ToA4P79+0hJSYGdnR1ycnKQn5+P\ntm3bcrJR26DSV0rOmjVrEBUVxfrxCxcuRGBgIFJSUuDq6gpjY2MsX74chw8frnScxGG5urrCyMiI\n6cp95coVTs4MADZs2IDdu3dj8eLFCAwMRFBQEF69esVq7LVr13D9+nW8ffsWGzduZI5L2gZVRdeu\nXQEAjx49wosXL2BqagoAuHDhAnOOLStWrEBaWhp69OjBPDePx2Pl0JKTk5GYmAg+ny/1Q5eXl1eu\nCW1VbNiwAUKhEPfu3cO8efOgoaEBNzc3BAYGVjqubE++CxcuyOTQyv54l+1HKAuenp44ffo0ZsyY\ngaCgIMTExOD8+fOcbKxbtw7nz59HQkICjh07Bnt7e/z+++84fvw4q/FOTk4wNTVFVFQU1q9fj7Nn\nz7Ju83TlyhVOc62M/fv3Iz4+HsnJybCzs0NJSQlWrFjxxUbElFKoQ1MC5s6d+8VzXLcyVFRUoKam\nhsuXL8PBwQHTpk3jtKKIi4vDhg0bmPvGxsbYvXs3pzkAQPv27SESiaCqqgo7OztYW1tj+fLlVY5r\n2bIlevXqhStXrqBXr17McU1NTVarK3t7ewCAv78/Tp48CTW10q/AlClTMG3aNE6vIT4+HmFhYZWu\n+L7Ey5cvcenSJfD5fFy4cIE5rqmpKfX+siEuLg5nz55l/o+NGzdGCYtWNbLMuzIb8tpTU1ODtrY2\nxGIxxGIxhgwZgs2bN3O2wePxEBERgalTp8Le3h4BAQGsx3/48AH29vbw8fGBnp4e9PT0YGdnx/Wl\n4OPHj0hJSZG6ONHV1WU9/vLlywgKCoKNjQ2A0s99fr587aBqA9ShKQGxsbHYvn076tevL3WcEILH\njx9zsqWmpoaQkBAEBQXh4MGDAEq7dLOlefPm+PPPPzFu3DgAwPnz59GsWTNOc9DQ0IBAIICOjg62\nbduGFi1aQCxm12+sR48e6NGjB8aNG8c4I1n4+PEjCgoKmK3WoqIizhcHXbt2RVZWFlq0aMH5+UeP\nHo3Ro0fj/v37GDRoEOfxZVFTU4NYLGYcSm5uLlRUqg6Pp6WlMRdLZW9LOHToUJU2EhISoKenB0II\n8vPzoaenB6D0s8nj8XD37l3Wr6Nhw4bIz8/HoEGD4OTkhCZNmpT7zFeFpqYm/vjjDwQHB+P48eMQ\ni8WcPt+Sz1SLFi0QFRWFFi1a4OPHj5zm4O/vDx8fH7x58wY9evTAo0eP0K9fP1bblhLq1KkDHo/H\n/E8l3e4plUNjaErArFmzMGvWLAwZMqTcualTp+LEiROsbb18+RJ+fn7o168fLCwskJaWhvDwcPzn\nP/9hNT4nJwf79u3D/fv3AZRedTo6OnLqvp2eno5mzZqhpKQE3t7e4PP5mDJlCtq3b1/lWEtLy0rP\nBwcHs5qDv78/Dh48CH19fRBCcOfOHcybNw/jx4+vcqzkhz8/Px8JCQno27cv6tSpw5xn4wi2bNlS\n6XkusbygoCBcvnwZ8fHxsLOzQ3h4OBwdHWFubl7puKqcjcQ5VUbZ2GFFsNkGllBQUIB69epBLBYj\nODgYfD4flpaW0NbWZm0jKysLISEh6NOnDwYNGoSMjAzcvXuX9S7E1atXMWjQIGRmZsLd3R35+flY\nsGABs8XOBktLSwQEBGDChAk4d+4cEhMTsXv3buzfv5+1jaNHjyIlJQXR0dGYM2cOAgMDYWFhwXkX\nobZBHVotpKioCBkZGejUqROncSKRCLt374aTk9M3m0N6enql59kG/4HSDL2HDx8CAPr164eWLVuy\nGqcIR+Dv71/pecnWKFtevHiBW7dugRCCoUOHolu3bpzGA0BJSQlevHiBli1bomnTpqzGFBcXQ01N\njXFcKSkpuH79On744QcYGxtznkN6ejpSUlIwdOhQFBYWQiQSQUtLi9VYkUiEX3/9Fb6+vpyfV5HY\n2dkhMDAQVlZW8Pf3h7q6OszNzTkltwClCV83b94EAAwbNoxT8lethVBqFZGRkWTMmDFk5MiRhBBC\n/vnnHzJnzhzW48ePH//N5/A52dnZRCwWs3rss2fPKv3jwrZt21gd+1rw+fxK/6rCxcWF/Pvvv4QQ\nQj59+kRMTU2JhYUFGTZsGAkODmY1h6lTp5KkpCRCCCEpKSlEV1eXuLq6EgcHB7Jz505Or+f06dPE\n1taW/Pzzz4QQQpKTk8n06dM52Zg+fTr59OkTpzFlSUpKItOnTyfm5uaEkNLPy4EDBzjZmD9/Pvn4\n8SPZu3cvmTJlCpk7dy6ZNWsWJxupqamkqKiIuV9YWEjS0tI42aiN0BhaLWP//v0ICAhgti50dHTw\n+vVr1uN79eoFR0dHjB07FhoaGsxxLlsyFc2hqpWXhIcPH2Lnzp1o1KgR5s+fD2dnZ+Tm5kIsFsPD\nwwOGhoaVjpckXAgEAjx79gxdunQBIQQvX75Ez549OSUQ3Lp1q9yx69evY8WKFVWO3bp1K1auXIkF\nCxZUmEzBZnvK3NxcKk1cYof8f/yqquzX2NhY5v0IDAxEhw4d4OXlhaysLMyePRsWFhZVzuHjx4/o\n2LEjAODs2bMwMzPDunXrIBAIYGdnh2XLllVpQ8KJEyfg7++PCRMmAAA6dOgglYnJhvr168PS0hJD\nhw6Vir+tXbuW1XgXFxc4OzvD1dUVQGnM1snJCfPnz2c9hwMHDgAozSgePHgw+Hw+hg8fzuFVlJZA\nlM1oVFFRYbKCKV+GOrRahpqaGho0aCB1jEt2Wn5+PjQ0NKRS9Xk8HieHVtEc2LJhwwYsW7YMfD4f\nv/zyCw4fPox+/fohMTERy5cvr9KhnTx5EgCwaNEiuLu7Q0dHB0BpcoMkSaYqTp48iVOnTiEtLU0q\nppefn4/+/fuzsmFmZgYAcHBwYPX4iuBaLvE5ZeN+t27dwtixYwGUJv7IQkxMDH777TcAgLq6Oues\nR3V1dairqzP3uSRzSBgzZgzGjBnDeZyEwsJC9O3bV+oYlzggIF0Y/+OPPwIA3r9/jzZt2rC2IRKJ\npN4LdXV1VpmrtR3q0JSAjIwMTl+GyujSpQuCg4MhEonw6tUr+Pr6sv4RBoDt27d/0zmIRCIMGzYM\nALB3717069cPANC5c2dOc0hMTGScGVB6Jf7y5UtWYy0tLWFoaIhdu3ZJlRpoamqicePGrGxIfjT1\n9fU5zPrL8Pl8pKamSqWJDxgwoNIxDRo0wNWrV9GyZUs8ePAAmzZtAlDqSIqKilg9b9euXbFjxw60\nbNkSr169YuI8fD6/wgLjytDV1cWhQ4dQVFSE6OhonDx5knMcTpLmLiva2tpITU1lnPGFCxc4O/iy\n9XzFxcV4/fo1OnbsyCmG1qRJE0RGRjIXihEREZySY2orNClECSirxrBw4ULs27dPZluFhYU4dOiQ\nVLB5/vz5qFu3bqXjEhMTkZqaipEjRwIAPDw8GMmrKVOmSDmHrzUHQPq9+FxRg4vCxuLFi9GoUSOm\n/CA4OBgfPnyQSekiOztbypFwufgYM2ZMhSuZixcvsrbh7+8Pb29vvH37Ft26dcOTJ0/Qr1+/KpMj\nkpOTsXHjRrx//x6//PILbG1tAQA3btxAdHQ0Vq5cWeVzFxQUwNvbG1lZWRg/fjxTGxgbG4uUlBTG\nJhvEYjECAgKkPhf29vacVnqvXr3Crl27yil9sFXpSEtLg4uLC+Li4tCwYUP8+OOP2LFjB6dko895\n+vQpTp48yVwwsCE1NRVOTk549+4dCCFo3bo1PDw8WGUC12q+YfyOwhIrK6sKb1cnc+fOJffu3WPu\njx07loSGhpKAgAAyf/581naEQiHZunWrzPPo0aMH6d+/P+nXrx/R0dEh/fv3Z+737NmTtZ3CwkJy\n+PBhMmfOHDJnzhxy+PBhqSA8GyIjI8no0aPJTz/9REaOHEm6d+9OzMzMONnIyspi/l6/fk2OHDlC\ndu/ezcmGhYUFKSwsJOPGjSOEEPLvv/8SR0dHTja+NUKhkCxbtkxuO5MmTSK3bt0iFhYW5PXr12Tv\n3r1kz549rMaKRCISGhpKCCEkPz+fVWINWywsLGQal5eXR/Ly8hQ2j5oO3XJUAhSpxjBjxgx4enoy\nBcUfP37EsmXLcPTo0UrHvX37VqoAuH79+kwciEugWlVVFbGxsTLMvJRnz57JPLYs9erVY+r7gNLX\nd+rUKfz666+sbezZs0duqabPi9JnzpwJW1tbLFmyhLUNdXV11KtXD0BpskvXrl2RnJzMaR7fGlVV\nVWRkZEAgEEjFjrhSXFzMbOP+8MMPWLhwIWxtbbF48eIqx6qoqODIkSMwMzPjXNBdlmPHjjG3xWIx\n/vnnH9bF9+fOnYOVlZWUjbLMmDFD5nnVBqhDUwISEhIwYMAAEEJQXFzM3AZKHRwXUd/c3FwpIeJG\njRohOzu7ynGfy+6UdWJsxpdFR0cHc+fOxdixY6V+OOQJ5svChw8fcPHiRYSEhCA9PZ1TYgugGKmm\nhIQE5jYhBPHx8ZyD/82bN8enT58wcuRIzJw5E40bN2ZdU/c90bZtW0yePBnGxsZSnwsuP+Lq6uoQ\ni8Vo3749jh8/zlkyaujQoTh69CjMzMyksnjZxkYB6e+KqqoqjIyMYGJiwmpsYWFhORsU9lCHpgQo\nalUClF6Flk0ySU9PZ7Xqa9GiBZ48eYI+ffpIHX/8+DFn6SuBQABtbW3cuXNH6nh1OLSCggJEREQg\nJCQEiYmJMDY2xqtXr3Djxg3OtiRSTbq6ujJLNZXVbVRVVcWPP/7IWRtTokyyZMkS3L59G3w+HyNG\njGA1ViwW48KFC8xqW1YuXbpU7v9X0bHKaNeuHdq1a8fIaMnC6tWrUVhYiLVr18LT0xN37tyBh4cH\n6/FhYWEAIKW+w1Up39HREQUFBRAKhVV2sficSZMmMcXkXHYLKP/Pt93xpLChoKCACAQC5n5iYiI5\nduwYuXTpEmdb165dI0ZGRsTJyYksX76cjBgxgly/fr3KcXFxcWTkyJHEy8uLREVFkaioKHLgwAEy\ncuRIEhcXx3ken/Po0SO5bbDhp59+InPmzCH3799njkkKvLmSn59PRCIRKSkpIWfOnCF///03ycnJ\nUdRUq6SoqIgIhULm/qtXr4iPjw+JiIjgZMfGxkbuuVhbWyvcblFREQkLC2P92Ozs7HLH379/qEwF\nQwAAIABJREFUzzk2+jnFxcWsH3v8+HEyYsQIoqenR/T09IihoSE5duwYIYSQd+/esbZjZ2fHdZoU\nQgjNclQCpk6dik2bNqFDhw5ISUmBvb09LC0t8fLlS/Tp04ezFFVOTg4ePXoEAPjpp59Y6zBmZWXB\n19cXL168AFCasu3g4CCTOC9QqisZEhKC0NBQNGjQAGfOnGE1Th6Jo6NHjyI0NBSqqqowNTXF2LFj\nMW3aNM69qj4nJycH2tranGKcsbGxaNiwIbp27YpLly7h3r17aNeuHSZOnMgqjuTg4AB3d3d07NgR\nqampGD9+PExNTZGUlIQBAwZg6dKlrOaxY8cOaGtry7TNduPGDdy4cQMhISFSNXl5eXlISEjgXAgs\nEolw8+ZNhISE4NatWxg4cCD27t1b5TgXFxcMHz683Irw8uXLuHnzJtavX89pHoQQxMTEIDg4GFFR\nURUW0X+Ol5cX4uLi4OrqyvQtS0tLY767d+/eZf0Z37x5M4RCYbn/SdkOE5TyUIemBFhaWjKiu3v2\n7MHHjx/h5ubGqDGwFeSVIE9ri8jISBgaGkoV5XLh9evXCA0NRUhICOrUqYP09HQEBgYyBahs+eWX\nX7B//36ZC7RfvXqFkJAQhIWFIS0tDUuXLsXo0aNZNVCUV60EANzd3Zl4Wbdu3ZCbm4vhw4cjLi4O\nqqqq2LZtW5U2yn4uPD09kZubK6XSwfZzUVGtF9tttn/++QdPnz6Fl5eXlJqGpqYmhgwZwvpi6e7d\nuwgJCcH169fRt29fxMbGIiIiQurHvDJsbW2/6Cy46Cg+fPgQISEhiIiIwMePH5megY0aNapyrImJ\nCc6fP1+u/KSoqAj6+vrYsWMH6zhtRSLEPB6Pk2J/bYTG0JSMmJgYJjNPFjUGeVtbXLhwARs3bsSQ\nIUNgamoKAwMD1koKEydORElJCcaOHQsvLy+0bdsWxsbGnJ0ZIL/EUYcOHeDo6AhHR0c8e/YMISEh\n+OWXX1g1aZRXrQQo/T+GhISguLgYRkZGiI6OhpqaGqZOncrUxnFBHpUOeRpT9uzZEz179sS4ceNY\n1RFWhKGhIbp06YLx48dj5cqVqF+/PoyNjVk7M+B/yRQVwaY10a5du3Dx4kW0bdsWpqamcHR0hK2t\nLadCbR6PV+F7UK9ePbRo0YJT0tG3FlhWVqhDUwK6d+8ODw8PtGzZEqmpqYwaw6dPnzjb8vHxYVpb\n+Pr6Mq0t2LJ9+3YIBAJERUXhzJkzcHNzw/Dhw1k1pWzatClevHiB7Oxs5OTkoG3btjKXIcgqcTR7\n9mwYGhrC0NCQKVLV0dGBjo4OKw1GQDFqJRKnU69ePbRp04bpw8Xj8Vj3eZNXpePSpUuVnufy/t65\ncweenp7IyMiASCTi1A/NxMQEV69eZRIyRo4cyflz0bRpUzx+/LicbNXjx49ZrRL9/f3Rs2dPTJ48\nGcOHD5fpYrFly5a4fft2OfWX27dvs846ffToEVxcXJCWloZu3bph8+bNnFVwajPUoSkBGzduhI+P\nD16/fo2//vqLuXJ9+fIlc1XOFnV1deYqUiAQoHPnzpxrltTV1fHzzz+jbt26TJYcG4fm5eUFPp+P\nS5cuwdPTEykpKfj06VOFP0Rf4s8//8TMmTNlljjauHEjrl+/jp07d+L169fo378/hg8fjiFDhjC1\nXFVRtnnm52PY/ghmZ2fDx8cHhBDmNlAau2EryLtx40Z4e3sjKSkJR44cYVaq//77L6tU96tXr1Z6\nnotDc3d3x549e9CtWzdWzUXLsmbNGqxevRp37txBaGgotm7dCj6fj7CwMBgZGUFTU7NKG87Ozliy\nZAlsbGyYOFN8fDyCgoJYXbDdvHkT0dHRCA0Nhbu7O/T09FBcXAyhUMj6AmPt2rWYP38+Bg4cKDWH\nBw8esNYJXb9+PX7//Xfo6uoiMjISmzdvrrJGlPI/aAytlrFgwQJs2bIFf//9N2JiYtCwYUMIhUIc\nPnyY1fjo6GiEhYXh9u3bGDBgAExNTZkrWq5kZ2cjPDwcISEhyMzMZCW2u2HDBsTGxsLV1RUDBw7k\n/JxlEYlEiIuLw/Xr13Hnzh1oaWlh2LBhVToDHR0daGhoMHWBEqdGCIFAIMDTp0+rfO49e/ZUep5L\nYfX3gIODA3x8fDg7s4ooKSlhEkNu3rxZrrzjS2RnZ+PEiRNM0lKXLl3g4ODAurebBIFAgKtXryI0\nNBT379+Hvr4+du7cyWpscXExgoODGV3Qzp07c9qOlUfOjUIdWq3m7t27TGsLtg5p0aJFMDMzw4gR\nI1ivaNiQnp7OWi/v6dOncHd3R6dOnTB58mSpH1F5ssDev3+PGzduyC1wqyysXLkSW7duBVDa+kWe\n1/3w4UMcOHAAgwcPlvosTZ8+Xa45FhUVKfRzxpW8vDxERESw7ngtLz///DN+//135r6Hh4fU/eoW\nH1A2qEOrJXz48KHS81yUEHJycvDkyRMAQJ8+fVhnsimSO3fuYNGiRVKdmdlkgW3evLnSbcFVq1Yp\nbI7fO9bW1ggKCgIg/0pg5syZqFOnDrp37y71/irbSvNbU9Xnb8uWLdU0E+WExtBqCba2tlLNIMvC\nRQnhzJkz8PT0hK6uLgghWLduHRYtWlRtq5rs7Gxs3boVaWlp+Pvvv9GjRw9O47t27QqgNPj+4sUL\nmJqaAijN3pScU1a46iDKqwtalrdv3yIkJERh9mor1GHJB12hKRHJyck4evQoMjIypJofVmdtiomJ\nCU6dOsWsynJycjB58mTW7U5EIhF8fX1llvUxNjbGnDlzMGHCBLl+kCdMmICTJ08yAX+BQIBp06bh\n9OnTMtvkikgkQkREBGudvy/x+PFjrFmzBnw+H1FRUUhISIC/vz9cXFwqHaevrw9zc3MQQhAWFgZz\nc3Op82xLIIDSDtxGRkYK6e9WWFjIKWX/a9n4lrx//x67du3Cu3fvcOTIEbx8+RJxcXGwt7f/1lP7\nrqErNCVi8eLFmDRpEiZMmCBX8P3SpUuIjY0Fj8fDoEGDMGrUKNZjGzduLFXM3KBBA07blaqqqggJ\nCZHZoQUEBChki/Pjx48oKChgtPaKioqq3JZVNKqqqvjjjz/kdmgbN27EoUOHsGDBAgClzUrZJFI4\nOzszt3v37i3XHAIDA+Ht7Q0NDQ3UqVOHU9q+hAcPHmDt2rUoKChgHLOfnx/WrVtXbTYsLS1hbm4O\nMzMztGvXjvXzSsZWBhcBhJUrV8LW1pbR6ezQoQOWLl1KHVoVUIemRKipqWHKlCly2Vi3bh1SU1OZ\nq/FTp04hOjoabm5ulY6TrAI7duyIiRMnYtSoUeDxeIiIiJCKY7FhwIAB2LBhg0yyPoqK182aNQvW\n1tbQ19cHIQR37tzBvHnzWI3t379/hatDyY84l+4HBgYG8Pb2LteyREtLi7UNsVhcLqGGzQWPIreJ\nY2Ji5LaxZcsWHD16lPk/9OjRA/fv369WG4cOHUJYWBiWLFkCHo8HMzMzmJqasmraKnE+iiA3Nxdm\nZmb4888/AZR+9xWRQVrToQ5NiRg5ciROnDiB0aNHS8VKuKyQYmJiEB4ezvwg29jYlNtqqghJbVSr\nVq3QqlUrCAQCAMDw4cM5b/1JugeU7Q5d3bI+9vb2MDQ0xMOHD8Hj8bB48WLWxa9xcXEKm4ekf1rZ\n/lc8Hg9RUVGsbbRu3RqPHz8Gj8djtnQ7dOigsDmy4cmTJ+jevTs0NDQQEhKCf/75B9OmTUPr1q05\n2fn88bL8iMtj44cffsDs2bMxe/ZsvHr1Cl5eXtixYwerjhfydLX+nPr16yM3N5f5bj18+FBmmbfa\nBHVoSoQkC61soSXX1hbt27dHRkYG8+XLzMxk1db9S9lqAoGAVf1YWRQp6yNPrCQhIYHJ1qxbty5r\nh6bIjFGu711FrFu3Dhs3bkRGRgaGDh2KoUOHctqmUwQuLi44f/48EhIScPjwYdja2sLZ2ZnT/7p1\n69Z48OABeDweSkpK4OPjw1klQxE20tPTERYWhvDwcKioqLBWkFHkyn3lypWYN28eUlNTMWnSJOTm\n5kpdAFIqhiaF1DIcHBzw5MkTRpnjyZMn6N27N7PFxWbbRCwW49atWwgNDcX169fx008/wcvLi/Uc\nBAIBLl68iPT0dKnkFkdHR9Y25I2V7N69Gw8ePICFhQUAIDQ0FP3792elUG9sbKyQjFEJiYmJSExM\nlBKLrioeI0EkEuHEiRNy1XvFxsaWK1Kv6FhlSNL+Dxw4gBYtWsDe3p5zKUBOTg42bdqE27dvgxAC\nAwMDrFmzBtra2tVmw97eHkKhEGPHjoWZmRkrseqvhVAoRHJyMggh6Nixo8yC4LUJ6tCUAIk+3Je0\n97gUW1YVpNfT0/viudjYWISEhCAqKgo6Ojp4+PAhLl++zEqaqCwzZ85EgwYN0KtXLylhYy4yXvb2\n9ti7dy/mzZvH1FJZWFiwTh23tLREUFAQ8/xCoRA2NjacOxfIi5eXF6Kjo5GUlIRhw4bh5s2bGDhw\nIPbv38/axvjx4xEQECDzHCpyPFyd0ZQpU2BsbAx/f3/4+vqiWbNmsLKyqvb3U16SkpLQqVMnhdjK\nzs6WukhhE4dTpL5mbYRuOSoB9+7dg76+/he197h8yPX09JCeno6UlBQMHToURUVFEAqFVSYhjBw5\nEu3bt4etrS2WL18OLS0tGBsbc3ZmQGnNkiL06eSNt+Tl5TFtQWTpkHzv3r0Kj7NtxQMA4eHhCAoK\ngo2NDbZv3453795xLu4eOHAgNm3aVC7Jpqoavbi4OMTFxSEnJ0cqhpeXlweRSMRpDrt378b58+ex\nfv16tGjRAhkZGZwzWTdu3FjumJaWFnr37s06E1dWG+fOnYOVlRWuXbtW4TYwG21MCZGRkfDw8MC7\nd+/QpEkTZGRkoHPnzqxa2ChSX7M2Qh2aErBo0SIAiim6/O9//4vTp0/j48ePiIiIwJs3b+Dm5oa/\n//670nEjR47EtWvXEBkZibp162LEiBEy14H1798fz58/R/fu3WUaD8gfK5k1axZsbGyYLMe7d+9i\n8eLFnOZQ1ikXFxfj8ePH6NWrF6fklrp160JVVRVqamrIy8tD8+bNkZ6ezmkekjhgWQ1JHo+HEydO\nVDqupKQEBQUFEIlEUg5dS0uLVVPNsrRs2RKzZ88GUNoFok2bNrCzs+Nko7i4GElJSRg7diyA0tXK\njz/+iISEBNy5cwdr1qz5ajYk7WdkubD5HE9PT5w+fRozZsxAUFAQYmJimOSfqqCF1XLylTtiUxTA\n77//ztw+c+aMXLbGjRtHiouLiZWVFXPMwsKC1ViRSERu3rxJVq5cSYYNG0b69etHLl68SAoKCjjN\nwdTUlPTq1YuMGTOGWFhYMH9cyM7OJsuWLSP6+vpkyJAhZPny5SQnJ4eTjTdv3pCLFy+Sixcvkjdv\n3nAaWxEZGRnE0dGR0xgXFxfy8eNHcvz4cTJmzBhia2tLVqxYIfdcuPD69WvmtkgkInw+n/XYR48e\nkV9++YUsWrSIPHv2jFhYWBA9PT2ir69Pbty4wWke9vb2RCgUMvdLSkrIhAkTiFAoJKamptVm43OO\nHTvG6fE2NjaEEEIsLS2JSCRibnPl6tWr5M8//yT79u1j/iiVQ1doSkBCQgJz28fHR676IXV1damU\n/7JJGVWhoqICAwMDGBgYQCAQ4Pr16wgJCYGLiwtrRXQArJX9K6NJkyasFdC/BJ/PR926dSEUChEf\nH4/4+HhOTRg/p1WrVkhMTOQ0RtJ2Z+rUqRg2bBjy8vI4Cyzn5eXBy8uLqbfS1dXFvHnzWNey7dq1\nC+vXr4eKigrGjx+PvLw8TJ8+nWkkWxnr16/HokWLwOfzMX36dBw8eBADBw7Eixcv4OzszPSNY4Ok\n2F2Snl5YWIgPHz5AVVWVtaSXImx8jre3N6ft04YNGyI/Px+6urpwcnJCkyZNpGoM2eDq6oqioiLc\nuXMH9vb2uHjxIvr06cNx5rUP6tCUAEVq7unq6uLQoUMoKipCdHQ0Tp48CWNjY8521NXVMWrUKIwa\nNQoFBQWsxsyePRsWFhYYNWqUTLG3ssgbb1m7di3i4+PRuXNnJvbG4/E4OTR3d3fmfyMWi/Hs2TP0\n7NmT9XgJoaGhSE1Nxbx585CZmYn4+HhOyh2rV69Gx44d4eHhAaA0HrR69WrW24YvX76ElpYWzp8/\nD0NDQyxfvhy2trasHJpIJIKRkREA4MCBA0xmZNeuXVk1GS3LrFmzYGVlhcGDB4MQgnv37mHu3Lko\nKChgLamlCBufw/V1eHl5oV69eli1ahWCg4PB5/MZFRe2xMXFITg4GJaWlnB0dMSMGTOYLV3Kl6EO\nTQl48+YNNm7cCEIIc7ssXDT3nJycEBAQgG7duuH06dMwMjKSW06H7dXnxIkTERYWhq1bt0JPTw8W\nFhYwMjKS6cpZ3nhLbGwswsLC5LpYKOt0VFVVYW5uzrlH24YNGyAUCnHv3j3MmzcPGhoacHNzQ2Bg\nIGsbKSkpUs5ryZIlsLKyYj1eKBSipKQEERERcHBwQJ06dVi/L2UfJ2uzUwn29vYwMjLC48ePAQBL\nly5lagPLtlD52jY+h8vrEIlEmDNnDnx9faGioiLzborkvdTQ0MDbt2/RuHFjZGVlyWSrNkEdmhKg\nKM09kUgEZ2dn7Ny5ExMmTFDE1DghWdEVFhbi6tWrCAoKwrp162BoaAgLCwsYGBiwtvX8+XOcOnWK\nSbufPHkypk6dipMnT7Kq4erTpw+Sk5NlStHOyMhAmzZtFCIdFRcXh7NnzzL9tho3boySkhJONurW\nrYuHDx+iX79+AEpVJdg2lARKLzSMjY3Ro0cP6OrqIj09nfV2ZUJCAvT09EAIQX5+PlP2QQhhvXL/\n/LW0aNECxcXFSE1NRWpqKqesUVltVFYUXTb1vipUVVWhoqICPp8vl7LHiBEj8OnTJ8yaNYtJrhk/\nfrzM9moL1KEpAZIfzvDwcKbdiYTw8HDWdlRVVZGRkcG5zciXIISgsLCQc3xAQ0MDZmZmMDMzQ0JC\nAlauXImgoCBW8kIS5I2V2Nvbw97eHq1atYK6ujqj5sCm9mrBggXM4xYuXIh9+/axnvfnqKmpQSwW\nMz+mubm5nMsP1q1bhxUrVkAgEIAQAg0NDWzbto31+OnTp0sVZrdp04Z1pmZ8fDynuVaGv78/fHx8\n8ObNG/To0QOPHj1Cv379OGWNympDkXJm9evXh6WlJYYOHSr13WCzk/L48WO0bt2a2aIsKChAt27d\n0KlTJ5kFvWsT1KEpEX/++Wc5h1bRscpo27YtJk+eDGNjY6kvG9s6G2dnZ7i6ukJVVRX29vb48OED\nZs6cyalO5/379wgPD0doaCiysrJgamrKdE5mi7yxktWrV2PLli3o1q0bZwdSNqaSlpbGaeznTJ06\nFQsXLkROTg727t2L8PBw1oopx48fh4ODAwQCAUJDQxlJLi7SWxXB4/Fw7tw5Vmn3ZQvj5cXHxwcB\nAQGYMGECfH19kZiYiN27d1e7DXkZM2aMzPVibm5uTE3gvXv3sGPHDri4uODZs2dwdXXlXE5R26AO\nTQm4du0arl+/jrdv30rFz/Ly8jj/oLRr1w7t2rVjtoi48vz5c2hpaSE4OBhDhw6Fk5MT7OzsWDm0\n//73vwgJCUFycjJMTEzg7OyMAQMGcJ4DIH+sRFtbW+YfnbJbU7LG4GbPng03NzdYW1ujV69euHXr\nFggh8PT0ZN29wN/fHw4ODli/fj3Onj0rtyMry759+zjXkcmLuro6s1UqEAjQuXNnJCcnV7sNebGx\nsUFRUREyMjI4b2mLRCLm/xgWFoaJEyfCxMQEJiYmnOKitRXq0JSAli1bonfv3rhy5YpUSrempiZn\nVQkueokVIRQKIRQKERkZicmTJ0NdXZ31CicuLg5z5syBvr6+QlphyBNv6d27N5ydnTFy5EipLUo2\nWY4JCQkYMGAAE1+ROGXCQYTW1tYWM2fOhLW1NWbNmiVTt+wOHTrA1NQUmZmZUvE8ttunlcUa379/\nz3k+8tKqVSt8+vQJo0aNwowZM9CwYUNWclGKtiEvV65cgYeHB0pKSnDlyhU8e/YMnp6erHVShUIh\n1NTUcPv2bbi7uzPnuKq31EaolqMSUVJSIrNA6aZNm7BmzRrMnTu3wvNsezl5e3vjyJEj6NatG44e\nPYrMzEwsX74cp06dkmlesiJvvKUiBXUej8cp9iQv+fn58PLywo0bN2BlZSXl5Nlu4b558wYzZ87E\nwYMHy52rqkHl0KFDcfToUabJqQRCCCZNmoSbN2+ymsPX4O7du+Dz+Rg+fLjM8V6uNkQiEX799Ve5\nu0HY2tri77//xrRp0zjrjB48eBDXrl2DtrY2MjMzcfbsWfB4PKSkpOD333+Hn5+fXHOr6dAVmhJx\n8+ZNeHp6IiMjA0KhkNOKQLJdwUUAuCJ+/fVXqeB069atq5TN+hrIGyvZvn37V5wdO+rUqQMNDQ0I\nBALk5+fLtGpt1aoVQkNDIRAIWLcCkjBixAjk5+dDR0en3LnBgwezsqGrq1tpyxS2HatFIhHMzc1x\n4cIFAJWLZH8tG4rKUFRTUys3nu3W9Lx586Cvr4+srCwYGBhI1Tm6uLjIPKfaAnVoSsTmzZuxb98+\ndO/enXPsRtLpWZYfirLk5eXh3LlzSE9Pl9oC4br1KS/yxkpSUlKwYcMGZGVlMX28rl27hjlz5nyt\nKUtx/fp1bN26FcbGxjh79qzMPd0AICoqClu2bJHa4tq/fz8OHDhQ6bjNmzd/8RxbFRZFdKoGSp1J\nx44dmZKIb2VDngxFCV26dEFwcDBEIhFevXoFX19f9O/fn/V4SflFWTp27Mh6fG2GOjQlolWrVujW\nrZtMiQiKSjWfPXs2evXqJfM8ACA1NZVJl79z5w6eP38Oa2vrcltflSFvrGTt2rVYunQpIz3VvXt3\nLF++vNoc2qFDh+Dp6SlT7Oxz9u7dC39/fyb1XkdHB6mpqXLbZcPnSUkfPnyQqtti2zQVKBU1Njc3\nR9++faUcPNvtcEXYkCdDUYKLiwsOHToEdXV1LF++HMOGDcP8+fPlsklhB3VoSoSTkxNmz54NPT09\nqZgAm3iLolLNi4qKOF2tVsTChQsRGBiIlJQUuLq6wtjYGMuXL+ek8ShZfSxcuBCDBw9mYiVsKSgo\nkMqw5PF4UFOrvq/DyZMnFWZLTU2N08XA10CySnzz5g0aN26Mt2/fokOHDsz2Hxu4djv4GjYUUSyf\nlJSEpUuXsmoWS1Es1KEpEZ6enqhfvz6Ki4s5q0koItUcAMzMzBAYGFguO5CtsgRQKnKspqaGy5cv\nw8HBAdOmTWOUMtigiHhL48aNkZaWxrwXERERaNasGWc73wOdO3dGWFgYxGIx0tLS4OvrW+G21ddk\n9+7dOHXqFH777TcEBQUhOjqaU9E/IP92uDw2IiIi8PbtW0ydOhVAaVlITk4OgNIEIonEGhu2bt2K\n9+/fw8TEBGZmZqzLMCjyQx2aEvHu3TvWHZk/RxGp5kBpqcCmTZvg6enJOAMej4eoqCjWc1FTU0NI\nSAiCgoKY7Dwuqv+KiJW4urpizZo1SEpKwsiRI9G8eXO51fu/FS4uLjhw4ABUVFTg6OiIYcOGVfvq\nQFVVFU2aNIFYLAYhBAYGBoxYMlsePnwId3d3JCUloaSkBCKRCBoaGqw/m/LYOHLkiFRSkUAgQEBA\nAAoLC7Fq1SpODs3X1xdZWVkIDw+Hq6sr8vPzYWpqSrcdqwHq0JQIQ0ND3Lx5k1NLDglcZKUq48iR\nI7h8+TKaNm0qs40tW7bAz88Pc+fORdu2bZGWloZx48ZxsiFvrKR9+/bw8fEBn88HIeSbb9nJQ/36\n9bFixYoKSxGqiwYNGiA/Px8DBw6Es7MzmjZtWk6suCo2bNiA3bt3Y/HixQgMDERQUBBevXpVLTZK\nSkqkOqAPHDgQ2tra0NbWZpp/cqF58+aYPn06Bg8ejCNHjsDLy4s6tGqA1qEpEf3790dhYSHU1dWh\npqbGeXWlCGbMmIFDhw5xEr/9GnwpHZztltPHjx/RqFEj5n54eDgOHjzIurPw90Bubi78/PzQsGFD\n2NraYseOHbh//z7atWsHZ2dntG3bttrmkpeXBw0NDYjFYgQFBSEvLw9WVlZMdi0bbG1tcebMGVha\nWiI4OBgAYG1tzdRyfU0bo0ePxuXLlys8N2rUKERERLCeQ2JiIsLCwnDx4kVoa2vD1NQUJiYmcl0E\nUthBV2hKhCIFVGVFS0sL1tbWGDJkiFQMjUva/qtXr7Br1y68fPlSKiMuMjKStQ15YiWSpJZ69eph\n/fr12Lt3L5o3b15hj7XvGScnJ3Tv3h1v376Fvb09LCwssH37dty/fx+rV6+Wu0CYC3/++SeWLVvG\naHwCpY1Dly1bxtqGpCZPR0cH27ZtQ4sWLSAWiznNQ1Ybffv2xX//+99yXSj8/PzQt29fTnNYvXo1\nzMzMcPToUU5ZnhT5oSs0JSI2NhY6OjqoX78+zp07h3/++Qe//PJLtUr7+Pv7V3icS0+1yZMnY9Gi\nRdi8eTMOHTqEM2fOQCwWc8pQkzVWYmlpCU9PT3Tq1AmPHz/GlClTsGfPHlZNQb83JKsQQghGjBiB\na9euMeesrKxw7ty5apuLjY1NOamtcePGcVrxpqeno1mzZigpKYG3tzf4fD6mTJnCqVhcVhvZ2dlY\nsGAB6tSpw8jLPX36FAKBAAcOHOCUMFRUVMSUTbRv3/6b72bUJugKTYlYt24dUwR87Ngx2Nvbw9nZ\nGcePH6+2OVTkuLi2ECkuLmYU8X/44QcsXLgQtra2nByarLESNTU1RjC2b9++aN++vVI6MwBMmQGP\nxyu3tacIrUw2+Pn5wc/PD69evZJKec/Pz5fSHWXDDz/8AKBUo1NWzVFZbTRt2hR+fn64ffs2Xr58\nCQAwMjLi1OVaKBRi165dOHPmDNq0aQNCCDIzM2Fra4ulS5fKLFtHYQ91aEqEmpoaeDwjZDGSAAAI\njUlEQVQeIiIiMHXqVNjb2yMgIKDanj8yMhJpaWno0qULhg0bhmfPnmHXrl148+YNE69gg7q6OsRi\nMdq3b4/jx4+jZcuWMin/t2/fHiKRCKqqqrCzs4O1tTWWL19e6Zjs7GwpvUc+ny91v2xfsO+dtLQ0\nODo6ghDC3AZKM1dfv35dLXMwNTWFvr4+du7cCScnJ+a4pqYm65iRIlLmFZV2r6+vz8mJlWXbtm3I\nz89HREQEU8aSl5cHDw8PeHh4yF2/Saka6tCUCE1NTfzxxx8IDg7G8ePHGWXu6sDV1RXJycno3bs3\nQkNDERAQgH///ReLFi3ilNIMlMYYCgsLsXbtWnh6eiImJoZziressRJbW1vmh66i+8pEWbUXBwcH\nqXOf3/9aNGrUCI0aNcLevXvx4sUL3L9/HwAwaNAg1g5NESnziky7l5WoqChcvHhRqs5TS0sL69at\n49SzkCI71KEpEbt370ZISAg2bdqE5s2bIyMjAzNnzqyW537w4AHOnTsHVVVVFBUVwcDAAJcvX+aU\nxSZBEmTX1NTEli1bZJrPtm3bQAiBq6srvL29kZmZyUrOa8mSJTI93/eIrCuJr8GJEydw4sQJZvt2\nyZIlmDJlCrNiqgxFpMwrOu1eFng8XoWiBaqqqnKJGVDYQx2aEtG8eXNG5ionJwetW7fmpLAhD3Xq\n1GF0++rVq4d27dpxdmZfal0jgYtmnyLiLRTF4efnB39/f2hqagIA5syZg0mTJrFyaJ8+fZK67+rq\nytxmu3pWhA156dy5M4KCgsp9J8+dO0fFhasJ6tCUgIcPH2Lnzp1o1KgR5s+fD2dnZ+Tm5kIsFsPD\nwwOGhoZffQ5JSUlM0J8QguTkZNjY2LBuJgmUvo7WrVvD3NwcP/30E2RJsFWkRBFFsZRNeuCSAKGI\nlHlFpt3LipubGxwdHREYGMgkxMTHx6OoqKjKzgcUxUDT9pUAW1tbLFu2DHw+H66urjh8+DD69euH\nxMRELF++nFPhqaxUpd5eVTNJoFSDMTo6GqGhoXj+/DmMjIxgYWHBSXF+0qRJ2L17N7O9ZGVlBW9v\nbyZWwqY3m0gkQkREBExMTFg/r7JACEFhYaFU65OviaS78uHDhxEaGsoo1UdERMDc3JzVlrgiUuYV\nmXYvL2UzJbt06fJdbQ3XeAjlu2fcuHHM7bFjx0qds7Kyqta57Ny5k9WxqiguLiaBgYFk8ODBxNfX\nl/U4W1tbqfvr169nbtvb27O2Y2Njw/qx3zsrVqwgfD6fFBQUEHNzc2JgYED++uuvanlua2tr5vaj\nR4/IX3/9Rf766y/y6NEjzrZu3bpFfHx8iI+PD7l165ZM81GEDYryQrcclYCyNUWf6+NVd7D5xo0b\n5dQfoqKiWCtCCAQCREVFISQkBOnp6Zg2bRpGjx7N+vkVFSsxMDCAt7c3zMzMpFYzXLoGfC88f/4c\nWlpaCA4OxtChQ+Hk5AQ7OztWbYXkhZTZ4Onbt69c23vypMwr0gZFeaEOTQmoTClfIBBUyxwUUUDr\n7OyMFy9ewNDQEI6OjjK11VBUrESiYHHs2DHmGNeuAd8LQqEQQqEQkZGRmDx5MtTV1autsDonJ0fq\nPfyc6nCqFIoE6tCUAEUp5cuDIgpoz58/Dw0NDaYtvQTCQWR59erVWLBgAYKDgyuMlbClrEyUsmNv\nb48RI0agW7du0NPTQ0ZGRrXF0MRisUxF8RTK14AmhVA483kBLZekDkWhiMB7YmIiEhMTpQSSLS0t\nFTbH6qCiBBdJwX1Z8eivRUUajhTKt4Ku0CickKeAVpHIGyvx8vJCdHQ0kpKSMGzYMNy8eRMDBw5U\nOoemqqqKP/74Q8qhqaioVIszAyBT6QWF8rWgKzQKJywtLeHn58cU0Obn52PSpEmctBy/BywtLREU\nFAQbGxucP38e7969w6pVq3D06NFvPTXO7Ny5E02bNv0mCS4fPnxA48aNv/rzUChsoCs0CmdkLaD9\nnqhbty5UVVWhpqaGvLw8NG/eHOnp6d96WjLxLRNcqDOjfE9Qh0ZhhaSAdty4cZgwYYJUAW11yW8p\nkp49e+LTp0+ws7ODnZ0dtLS0qk1RQtHUpAQXCkUe6JYjhRVlg/+PHz9GbGwsgFIRWGV1BBJSUlKQ\nl5fHuX/X90RNSHChUOSFrtAorFBkAe33QmhoKFJTUzFv3jxkZmYiPj4evXv3/tbT4kxNSXChUOSF\nOjQKK2paAe2GDRsgFApx7949zJs3DxoaGnBzc0NgYOC3nhpnwsPDmQSX7du3MwkuFEptgzo0Citq\nWgFtXFwczp49y8T/GjdujJKSkm88K9moSQkuFIo8UIdGYUXz5s1rVN8xNTU1iMViRgszNze32uSi\nFE1NSnChUOSBJoVQWGFtbV0tbWqqi6CgIFy+fBnx8fGws7NDeHg4HB0dYW5u/q2nJhc1IcGFQpEV\n6tAorKgpBbSzZ8+Gm5sbfvzxR7x48QK3bt0CIQRDhw6VSSz5e+HzBJfs7GylTHChUOSBOjRKrSI8\nPBx79uyBtbU1Zs2apbSF4WUpm+ASHh6ODx8+YObMmUqZ4EKhyAONoVFqFaampjA0NISXlxfs7Oxg\nZWUlFTtTtmxNoGYluFAo8kAdGqXWUadOHWhoaEAgECA/P19pk0Ek1KQEFwpFHqhDo9Qqrl+/jq1b\nt8LY2Bhnz56FhobGt56S3EydOhULFy5ETk4O9u7dyyS4UCi1DRpDo9QqpkyZgvXr13+THm6KpqYm\nuFAoskIdGoWipNTEBBcKRR6oQ6NQlJj8/Hx4eXnhxo0bNSLBhUKRBxpDo1CUmJqW4EKhyAN1aBSK\nklITE1woFHmgDo1CUVIOHToET0/PGpHgQqEoAhpDo1AoFEqNgG64UygUCqVGQB0ahUKhUGoE1KFR\nKBQKpUZAHRqFQqFQagTUoVEoFAqlRvB/kWmMJ0U1fVAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5e0ced0320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "sns.heatmap(transform_df.corr())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The heatmap above shows us that:\n",
    "\n",
    "1. `Gr Liv Area` and `TotRms AbvGrd` are strongly correlated.\n",
    "2. `Garage Area` and `Garage Cars` are strongly correlated.\n",
    "\n",
    "The documentation tells us that these features reflect very similar information. \n",
    "\n",
    "We'll therefore drop `TotRms AbvGrd` and `Garage Cars`, because `Gr Liv Area` and `Garage Area` are continuous variables that capture more nuance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2928, 14)"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform_df = transform_df.drop(['Garage Cars', 'TotRms AbvGrd'], axis = 1)\n",
    "\n",
    "transform_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Categorical Columns\n",
    "\n",
    "Some columns from the original dataframe should be converted to the categorical type. \n",
    "\n",
    "We'll list these below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nominal_features = [\"PID\", \"MS SubClass\", \"MS Zoning\", \"Street\", \"Alley\", \"Land Contour\", \"Lot Config\", \"Neighborhood\", \n",
    "                    \"Condition 1\", \"Condition 2\", \"Bldg Type\", \"House Style\", \"Roof Style\", \"Roof Matl\", \"Exterior 1st\", \n",
    "                    \"Exterior 2nd\", \"Mas Vnr Type\", \"Foundation\", \"Heating\", \"Central Air\", \"Garage Type\", \n",
    "                    \"Misc Feature\", \"Sale Type\", \"Sale Condition\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're interested in which of these columns are still with us in our `transform_df` dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform_df_cat = []\n",
    "for col in nominal_features:\n",
    "    if col in transform_df.columns:\n",
    "        transform_df_cat.append(col)\n",
    "        \n",
    "len(transform_df_cat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that none of these columns are with us anymore. We'll now update the `select_features` function to include the above code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Updating `select_features` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def transform_features(df):\n",
    "    # Removing columns with more than 15% missing values\n",
    "    df_null_count = df.isnull().sum()\n",
    "    df_null_count = df_null_count[df_null_count < 440] \n",
    "    df_filtered = df[df_null_count.index]\n",
    "    \n",
    "    # Modifying Numeric Columns\n",
    "    numeric_filtered = df_filtered.select_dtypes(include=['float', 'int'])\n",
    "    numeric_filtered = numeric_filtered.fillna(numeric_filtered.mean())\n",
    "    \n",
    "    # Modifying Text Columns\n",
    "    text_filtered = df_filtered.select_dtypes(include=['object'])\n",
    "    text_filtered = text_filtered.dropna(axis=1)\n",
    "    \n",
    "    # Concatenating Numeric and Text columns\n",
    "    df_new = pd.concat([text_filtered, numeric_filtered], axis = 1)\n",
    "    \n",
    "    # Creating new columns, and dropping the 2 rows that had negative values there\n",
    "    df_new['Years Before Sale'] = df_new['Yr Sold'] - df_new['Year Built']\n",
    "    df_new['Years Since Remod'] = df_new['Year Remod/Add'] - df_new['Year Built']\n",
    "    df_new = df_new[df_new['Years Before Sale'] >= 0]\n",
    "    df_new = df_new[df_new['Years Since Remod'] >= 0]\n",
    "    \n",
    "    # Dropping unnecessary columns:\n",
    "    df_new = df_new.drop([\"PID\", \"Order\"], axis=1)\n",
    "    df_new = df_new.drop([\"Mo Sold\", \"Sale Condition\", \"Sale Type\", \"Yr Sold\"], axis=1)\n",
    "\n",
    "    return df_new\n",
    "\n",
    "def select_features(df):\n",
    "    df_numeric = df.select_dtypes(include=['int', 'float'])\n",
    "    df_corr = df_numeric.corr()['SalePrice'].abs().sort_values()\n",
    "    \n",
    "    strong_corr = df_corr[df_corr > 0.4]\n",
    "\n",
    "    new_df = df[strong_corr.index]\n",
    "    new_df = new_df.drop(['Garage Cars', 'TotRms AbvGrd'], axis = 1)\n",
    "    \n",
    "    return new_df\n",
    "\n",
    "def train_and_test(df):\n",
    "    train = df.iloc[:1460]\n",
    "    test = df.iloc[1460:]\n",
    "    \n",
    "    # Selecting only numeric columns, other than SalePrice    \n",
    "    numeric_train = train.select_dtypes(include=['integer', 'float'])\n",
    "    numeric_test = test.select_dtypes(include=['integer', 'float'])    \n",
    "    \n",
    "    numeric_features = numeric_train.columns.drop('SalePrice')\n",
    "    \n",
    "    # Instantiating model:\n",
    "    lr = linear_model.LinearRegression()\n",
    "    \n",
    "    # Training:\n",
    "    lr.fit(numeric_train[numeric_features], numeric_train['SalePrice'])\n",
    "    \n",
    "    # Predicting:\n",
    "    train_predictions = lr.predict(numeric_test[numeric_features])\n",
    "    \n",
    "    # RMSE:\n",
    "    rmse = mean_squared_error(train_predictions, numeric_test['SalePrice']) ** 0.5\n",
    "    \n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now test the above modified `select_features` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "37974.776072336594"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"AmesHousing.tsv\", delimiter=\"\\t\")\n",
    "transform_df = transform_features(df)\n",
    "filtered_df = select_features(transform_df)\n",
    "rmse = train_and_test(filtered_df)\n",
    "\n",
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the select_features function significantly improved the rmse.\n",
    "\n",
    "Next, we'll use simple validation to calculate the rmse.\n",
    "\n",
    "## Cross Validation\n",
    "\n",
    "The `train_and_test` function will be modified to accept a parameter `k` which controls the type of cross validation that occurs, which accepts integer values, but has a default value of 0.\n",
    "\n",
    "When k = 0 , holdout validation will be performed, which is what the function already does. \n",
    "\n",
    "When k is 1, simple cross validation is performed, and then the rmse will be returned.\n",
    "\n",
    "When k > 1, k-fold cross validation will be performed using k number of folds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_and_test(df, k=0):\n",
    "    \n",
    "    # Identifying numeric columns, and setting them as features:\n",
    "    numeric_df = df.select_dtypes(include=['integer', 'float'])\n",
    "    features = numeric_df.columns.drop(\"SalePrice\")\n",
    "    \n",
    "    # Instantiating the model:\n",
    "    lr = linear_model.LinearRegression()\n",
    "    \n",
    "    if k == 0: \n",
    "        # This is simply Holdout Validation\n",
    "        train = df.iloc[:1460]\n",
    "        test = df.iloc[1460:]\n",
    "    \n",
    "        # Training:\n",
    "        lr.fit(train[features], train['SalePrice'])\n",
    "\n",
    "        # Predicting:\n",
    "        predictions = lr.predict(test[features])\n",
    "\n",
    "        # RMSE:\n",
    "        rmse = mean_squared_error(predictions, test['SalePrice']) ** 0.5\n",
    "    \n",
    "        return rmse\n",
    "    \n",
    "    elif k == 1: \n",
    "        \n",
    "        # This is Simple Cross Validation, which means we do the test twice, \n",
    "        # the second time being with the train and test sets swapped.\n",
    "        \n",
    "        \n",
    "        # First let's shuffle the dataframe:\n",
    "        shuffled_df = df.sample(frac=1) # frac = 1 means returning the whole dataframe in a shuffled manner \n",
    "        \n",
    "        # Splitting into two sets:\n",
    "        train = shuffled_df[:1460]\n",
    "        test = shuffled_df[1460:]\n",
    "        \n",
    "        # Training on train set, testing on test set.\n",
    "        lr.fit(train[features], train[\"SalePrice\"])\n",
    "        predictions_one = lr.predict(test[features])        \n",
    "        \n",
    "        mse_one = mean_squared_error(test[\"SalePrice\"], predictions_one)\n",
    "        rmse_one = np.sqrt(mse_one)\n",
    "        \n",
    "        # Here we Swap - Training on test set, testing on train set.\n",
    "        lr.fit(test[features], test[\"SalePrice\"])\n",
    "        predictions_two = lr.predict(train[features])        \n",
    "       \n",
    "        mse_two = mean_squared_error(train[\"SalePrice\"], predictions_two)\n",
    "        rmse_two = np.sqrt(mse_two)\n",
    "        \n",
    "        print(rmse_one)\n",
    "        print(rmse_two)\n",
    "        \n",
    "        # Calculating and returning average rmse:\n",
    "        avg_rmse = np.mean([rmse_one, rmse_two])\n",
    "        return avg_rmse\n",
    "        \n",
    "    else: # k-fold cross validation, where k represents number of splits\n",
    "        kf = KFold(n_splits=k, shuffle=True)\n",
    "        rmse_vals = []\n",
    "        \n",
    "        # Split dataframe and iterate over each train and test set.\n",
    "        for train_index, test_index, in kf.split(df):\n",
    "            train = df.iloc[train_index]\n",
    "            test = df.iloc[test_index]\n",
    "            lr.fit(train[features], train[\"SalePrice\"])\n",
    "            predictions = lr.predict(test[features])\n",
    "            rmse = (mean_squared_error(test[\"SalePrice\"], predictions)) ** 0.5\n",
    "            print(rmse)\n",
    "            rmse_vals.append(rmse)\n",
    "\n",
    "        avg_rmse = np.mean(rmse_vals)\n",
    "        return avg_rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we'll repeat the predictions using k-fold validation now, with k = 4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "33272.63892241869\n",
      "35593.68719122992\n",
      "37839.5929174513\n",
      "28708.35741897865\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "33853.56911251964"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"AmesHousing.tsv\", delimiter=\"\\t\")\n",
    "transform_df = transform_features(df)\n",
    "filtered_df = select_features(transform_df)\n",
    "rmse = train_and_test(filtered_df, k=4) # Specifying k = 4\n",
    "\n",
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We observe that we managed to decrease the rmse to the 32,000-35,000 range with k-fold cross validation."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}