{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Model evaluation & cross-validation\n", "\n", "[José C. García Alanis (he/him)](https://github.com/JoseAlanis) \n", "Research Fellow - Child and Adolescent Psychology at [Uni Marburg](https://www.uni-marburg.de/de) \n", "Member - [RTG 2271 | Breaking Expectations](https://www.uni-marburg.de/en/fb04/rtg-2271), [Brainhack](https://brainhack.org/)\n", "\n", "\"logo\" \"logo\"   @JoiAlhaniz \n", "\n", "\n", "\"logo\"\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Aim(s) of this section\n", "\n", "As mention in the previous section, it is not sufficient to apply these methods to learn somthing about the nature of our data. It is always necessary to assess the quality of the implemented model. The goal of these section is to look at ways to estimate the generalization accuracy of a model on future (e.g.,unseen, out-of-sample) data.\n", "\n", "In other words, at the end of these sections you should know:\n", "- 1) different techniques to evaluate a given model\n", "- 2) understand the basic idea of cross-validation and different kinds of the same\n", "- 3) get an idea how to assess the significance (e.g., via permutation tests)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Prepare data for model\n", "\n", "Lets bring back our example data set (you know the song ...)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "There are 155 samples and 2016 features\n" ] } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "\n", "# get the data set\n", "data = np.load('MAIN2019_BASC064_subsamp_features.npz')['a']\n", "\n", "# get the labels\n", "info = pd.read_csv('participants.csv')\n", "\n", "\n", "print('There are %s samples and %s features' % (data.shape[0], data.shape[1]))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now let's look at the labels" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "scrolled": false, "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
participant_idAgeAgeGroupChild_AdultGenderHandedness
0sub-pixar12327.06AdultadultFR
1sub-pixar12433.44AdultadultMR
2sub-pixar12531.00AdultadultMR
3sub-pixar12619.00AdultadultFR
4sub-pixar12723.00AdultadultFR
\n", "
" ], "text/plain": [ " participant_id Age AgeGroup Child_Adult Gender Handedness\n", "0 sub-pixar123 27.06 Adult adult F R\n", "1 sub-pixar124 33.44 Adult adult M R\n", "2 sub-pixar125 31.00 Adult adult M R\n", "3 sub-pixar126 19.00 Adult adult F R\n", "4 sub-pixar127 23.00 Adult adult F R" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "info.head(n=5)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "We'll set `Age` as target\n", "- i.e., well look at these from the `regression` perspective" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "scrolled": true, "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "count 155.000000\n", "mean 10.555189\n", "std 8.071957\n", "min 3.518138\n", "25% 5.300000\n", "50% 7.680000\n", "75% 10.975000\n", "max 39.000000\n", "Name: Age, dtype: float64" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# set age as target\n", "Y_con = info['Age']\n", "Y_con.describe()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Next:\n", "- we need to divide our input data `X` into `training` and `test` sets" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "# import necessary python modules\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.svm import SVC\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.metrics import accuracy_score\n", "\n", "# split the data\n", "X_train, X_test, y_train, y_test = train_test_split(data, Y_con, random_state=0)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now lets look at the size of the data sets" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "N used for training: 116 | N used for testing: 39\n" ] } ], "source": [ "# print the size of our training and test groups\n", "print('N used for training:', len(X_train),\n", " ' | N used for testing:', len(X_test))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "**Question:** Is that a good distribution? Does it look ok?\n", "\n", "- Why might this be problematic (hint: what do you know about groups (e.g., `Child_Adult`) in the data." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "scrolled": true, "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAATsUlEQVR4nO3dfbBcBXmA8eclCUkEOgnhTgpJaOLHIBTbUCLlazoWayfaD6JNQcbadAYbHLCDY7Wi/aPa1hmsVgWnilSpsVUBRQpSxAIGrQNCLxjkKxJQMBeBhGgQOhMl8PaPPRcuMbm5CXv23Xv3+c3cye45+/HeM+Th5Ozu2chMJEm9t0/1AJI0qAywJBUxwJJUxABLUhEDLElFplcPMBHLly/Pa665pnoMSdpTMd7KSbEH/Nhjj1WPIEldNykCLElTkQGWpCIGWJKKTIoX4SRNTk899RQjIyNs27atepRWzZo1i4ULFzJjxow9up8BltSakZERDjjgABYvXkzEuG8ImLQyky1btjAyMsKSJUv26L4egpDUmm3btjFv3rwpG1+AiGDevHl7tZdvgCW1airHd9Te/o4GWJKKGGBJPbNg0aFERNd+Fiw6dNzn27p1K5/4xCf2eM7Xve51bN26dS9/y4nzRThJPfPjkY2c+qkbu/Z4l5xx/LjrRwN85plnPm/59u3bmT591/m7+uqruzLf7hhgSVPWOeecw/3338/SpUuZMWMGs2bNYu7cuaxfv557772XFStWsHHjRrZt28bZZ5/N6tWrAVi8eDHDw8M8+eSTvPa1r+XEE0/kxhtvZMGCBVxxxRXMnj27K/N5CELSlHXuuefykpe8hHXr1vGhD32I2267jfPOO497770XgIsuuohbb72V4eFhzj//fLZs2fJLj7FhwwbOOuss7rrrLubMmcNll13WtfncA5Y0MI455pjnvVf3/PPP5/LLLwdg48aNbNiwgXnz5j3vPkuWLGHp0qUAHH300TzwwANdm2dK7wF3+4D/RA76S+pf++2337OXb7jhBq677jpuuukmbr/9do466qidvpd35syZz16eNm0a27dv79o8U3oPuNsH/GH3B/0l9Y8DDjiAJ554YqfrHn/8cebOncuLXvQi1q9fz3e+850eTzfFAyypvxyycFFXd2IOWbho3PXz5s3jhBNO4Mgjj2T27NnMnz//2XXLly/nggsu4PDDD+ewww7j2GOP7dpcE2WAJfXMQxt/1PPn/MIXvrDT5TNnzuRrX/vaTteNHuc96KCDuPPOO59d/s53vrOrs03pY8CS1M8MsCQVMcCSWpWZ1SO0bm9/RwMsqTWzZs1iy5YtUzrCo+cDnjVr1h7f1xfhJLVm4cKFjIyMsHnz5upRWjX6jRh7ygBLas2MGTP2+FsiBomHICSpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkoq0HuCImBYR342Iq5rrSyLi5oi4LyIuiYh9255BkvpRL/aAzwbuGXP9g8BHM/OlwE+B03swgyT1nVYDHBELgT8APt1cD+Ak4MvNTdYAK9qcQZL6Vdt7wB8D/gZ4prk+D9iamdub6yPAgpZnkKS+1FqAI+IPgU2Zeete3n91RAxHxPDmzZu7PJ0k1WtzD/gE4I8j4gHgYjqHHs4D5kTE9OY2C4GHdnbnzLwwM5dl5rKhoaEWx5SkGq0FODPfk5kLM3Mx8EbgG5n5JmAtsLK52SrgirZmkKR+VvE+4HcD74iI++gcE/5MwQySVG767m/ywmXmDcANzeUfAMf04nklqZ/5SThJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAZakIgZYkooYYEkqYoAlqYgBlqQiBliSirQW4IiYFRG3RMTtEXFXRLy/Wb4kIm6OiPsi4pKI2LetGSSpn7W5B/xz4KTM/E1gKbA8Io4FPgh8NDNfCvwUOL3FGSSpb7UW4Ox4srk6o/lJ4CTgy83yNcCKtmaQpH7W6jHgiJgWEeuATcC1wP3A1szc3txkBFjQ5gyS1K9aDXBmPp2ZS4GFwDHAyyd634hYHRHDETG8efPmtkaUpDI9eRdEZm4F1gLHAXMiYnqzaiHw0C7uc2FmLsvMZUNDQ70YU5J6qs13QQxFxJzm8mzgNcA9dEK8srnZKuCKtmaQpH42ffc32WsHA2siYhqd0F+amVdFxN3AxRHxj8B3gc+0OIMk9a3WApyZ3wOO2snyH9A5HixJA81PwklSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklRkQgGOiBMmskySNHET3QP++ASXSZImaPp4KyPiOOB4YCgi3jFm1a8A09ocTJKmunEDDOwL7N/c7oAxy38GrGxrKEkaBOMGODO/CXwzIj6bmQ/2aCZJGgi72wMeNTMiLgQWj71PZp7UxlCSNAgmGuAvARcAnwaebm8cSRocEw3w9sz8ZKuTSNKAmejb0L4aEWdGxMERceDoT6uTSdIUN9E94FXNn+8asyyBF3d3HEkaHBMKcGYuaXsQSRo0EwpwRPz5zpZn5ue6O44kDY6JHoJ45ZjLs4BXA7cBBliS9tJED0H81djrETEHuLiNgSRpUOzt6Sj/D/C4sCS9ABM9BvxVOu96gM5JeA4HLm1rKEkaBBM9BvzhMZe3Aw9m5kgL80jSwJjQIYjmpDzr6ZwRbS7wizaHkqRBMNFvxDgFuAX4U+AU4OaI8HSUkvQCTPQQxN8Cr8zMTQARMQRcB3y5rcEkaaqb6Lsg9hmNb2PLHtxXkrQTE90DviYivg58sbl+KnB1OyNJ0mDY3XfCvRSYn5nviog3ACc2q24CPt/2cJI0le1uD/hjwHsAMvMrwFcAIuIVzbo/anE2SZrSdnccd35m3rHjwmbZ4lYmkqQBsbsAzxln3ewuziFJA2d3AR6OiL/ccWFEvAW4tZ2RJGkw7O4Y8NuByyPiTTwX3GXAvsDrW5xLkqa8cQOcmY8Cx0fE7wJHNov/KzO/0fpkkjTFTfR8wGuBtS3PMjnsM52I6OpDHrJwEQ9t/FFXH1NS/5voBzE06pntnPqpG7v6kJeccXxXH0/S5ODHiSWpSGsBjohFEbE2Iu6OiLsi4uxm+YERcW1EbGj+nNvWDJLUz9rcA94O/HVmHgEcC5wVEUcA5wDXZ+bLgOub65I0cFoLcGY+nJm3NZefAO4BFgAnA2uam60BVrQ1gyT1s54cA46IxcBRwM10Pt78cLPqEWB+L2aQpH7TeoAjYn/gMuDtmfmzsesyM3nuyz53vN/qiBiOiOHNmze3PaYk9VyrAY6IGXTi+/nmbGoAj0bEwc36g4FNO7tvZl6Ymcsyc9nQ0FCbY0pSiTbfBRHAZ4B7MvMjY1ZdCaxqLq8CrmhrBknqZ21+EOME4M3AHRGxrln2XuBc4NKIOB14kM6XfErSwGktwJn5bWBXn9l9dVvPK0mThZ+Ek6QiBliSihhgSSpigCWpiAGWpCIGWJKKGGBJKmKAJamIAe4HzffMdfNnwaJDq38rSbvhd8L1A79nThpI7gFLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBVpLcARcVFEbIqIO8csOzAiro2IDc2fc9t6fknqd23uAX8WWL7DsnOA6zPzZcD1zXVJGkitBTgzvwX8ZIfFJwNrmstrgBVtPb8k9bteHwOen5kPN5cfAebv6oYRsToihiNiePPmzb2ZTpJ6qOxFuMxMIMdZf2FmLsvMZUNDQz2cTJJ6o9cBfjQiDgZo/tzU4+eXpL7R6wBfCaxqLq8Crujx80tS32jzbWhfBG4CDouIkYg4HTgXeE1EbAB+r7kuSQNpelsPnJmn7WLVq9t6TkmaTPwknCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcBT1T7TiYiu/ixYdGj1b1VqwaJD3abqqtZOR6liz2zn1E/d2NWHvOSM47v6eJPNj0c2uk3VVe4BS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSEQMsSUUMsCQVMcCSVMQAS1IRAyxJRQywJBUxwJJUxABLUhEDLElFDLCkCfE78brP74STNCF+J173uQcsSUUMsCQVMcCSVMQAS1IRX4TTlLRg0aH8eGRj9Ri7t890IqKrDzltxkyefurnXX3M1rTw+x+ycBEPbfxRVx+zLQZYU9KkecX+me2tzNntxxx93K5r6fefLDwEIUlFDLAkFTHAklTEAEtSEQMsSUUMsKSppXlr22Q4aZBvQ5M0tUyit7a5ByxJRQywJBUxwJJUxABLUhEDLElFDLAkFTHAklTEAEtSkZIAR8TyiPh+RNwXEedUzCBJ1Xoe4IiYBvwL8FrgCOC0iDii13NIUrWKPeBjgPsy8weZ+QvgYuDkgjkkqVRkZm+fMGIlsDwz39JcfzPw25n5th1utxpY3Vw9DPh+j0Y8CHisR8/VLc7cO5NxbmfujZ3N/FhmLt/VHfr2ZDyZeSFwYa+fNyKGM3NZr5/3hXDm3pmMcztzb+zNzBWHIB4CFo25vrBZJkkDpSLA/wu8LCKWRMS+wBuBKwvmkKRSPT8EkZnbI+JtwNeBacBFmXlXr+cYR88Pe3SBM/fOZJzbmXtjj2fu+YtwkqQOPwknSUUMsCQVMcBjRMQDEXFHRKyLiOHqeXYmIi6KiE0RceeYZQdGxLURsaH5c27ljDvaxczvi4iHmm29LiJeVznjjiJiUUSsjYi7I+KuiDi7Wd6323qcmft9W8+KiFsi4vZm7vc3y5dExM3NKQsuaV607wvjzPzZiPjhmG29dNzH8RjwcyLiAWBZZvbtG8Aj4neAJ4HPZeaRzbJ/An6Smec259aYm5nvrpxzrF3M/D7gycz8cOVsuxIRBwMHZ+ZtEXEAcCuwAvgL+nRbjzPzKfT3tg5gv8x8MiJmAN8GzgbeAXwlMy+OiAuA2zPzk5Wzjhpn5rcCV2XmlyfyOO4BTzKZ+S3gJzssPhlY01xeQ+cvXd/Yxcx9LTMfzszbmstPAPcAC+jjbT3OzH0tO55srs5ofhI4CRgNWb9t613NvEcM8PMl8N8RcWvzUejJYn5mPtxcfgSYXznMHnhbRHyvOUTRN/+U31FELAaOAm5mkmzrHWaGPt/WETEtItYBm4BrgfuBrZm5vbnJCH32P5MdZ87M0W39gWZbfzQiZo73GAb4+U7MzN+ic6a2s5p/Ok8q2TmmNBmOK30SeAmwFHgY+OfSaXYhIvYHLgPenpk/G7uuX7f1Tmbu+22dmU9n5lI6n4w9Bnh57US7t+PMEXEk8B46s78SOBAY9/CUAR4jMx9q/twEXE7nP4TJ4NHm+N/occBNxfPsVmY+2vwH/Azwr/Thtm6O7V0GfD4zv9Is7uttvbOZJ8O2HpWZW4G1wHHAnIgY/bBY356yYMzMy5vDQJmZPwf+jd1sawPciIj9mhcuiIj9gN8H7hz/Xn3jSmBVc3kVcEXhLBMyGrHG6+mzbd28yPIZ4J7M/MiYVX27rXc18yTY1kMRMae5PBt4DZ3j12uBlc3N+m1b72zm9WP+5xx0jlmPu619F0QjIl5MZ68XOh/R/kJmfqBwpJ2KiC8Cr6Jz6rtHgb8D/hO4FDgUeBA4JTP75kWvXcz8Kjr/JE7gAeCMMcdWy0XEicD/AHcAzzSL30vnmGpfbutxZj6N/t7Wv0HnRbZpdHYKL83Mv2/+Tl5M55/y3wX+rNmzLDfOzN8AhoAA1gFvHfNi3S8/jgGWpBoegpCkIgZYkooYYEkqYoAlqYgBlqQiBlhTXkSsiIiMiL7/dJUGiwHWIDiNztmqTqseRBrLAGtKa86LcCJwOp0vgCUi9omIT0TE+uacvldHxMpm3dER8c3mhExf3+FTZFJXGWBNdScD12TmvcCWiDgaeAOwGDgCeDOd8w6Mnkfh48DKzDwauAjou09Dauro+bciSz12GnBec/ni5vp04EvNyWkeiYi1zfrDgCOBazsf5WcanbOHSa0wwJqyIuJAOif1fkVEJJ2gJs+d8+OX7gLclZnH9WhEDTgPQWgqWwn8e2b+WmYuzsxFwA/pfDvHnzTHgufTOTEQwPeBoYh49pBERPx6xeAaDAZYU9lp/PLe7mXAr9L5hoW7gf8AbgMez8xf0In2ByPidjpnszq+Z9Nq4Hg2NA2kiNi/+ULFecAtwAmZ+Uj1XBosHgPWoLqqOaH2vsA/GF9VcA9Ykop4DFiSihhgSSpigCWpiAGWpCIGWJKK/D8WGBmJv4D0egAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVvUlEQVR4nO3df7RdZX3n8fc3JBgxGWKSCzW/JtgKowMt4KUj4LQFSo3WBdKhIIJDOzhhxh+jowuEUerorK5F13QwXe1IyWCKLSwEEVctWEyKqdiFhV5ikPDDEqvVy69cYFCKAxj4zh9np73e3iQ3h3P299x736+1zrrn7L3Pfr73yc0nT56793MiM5EktW9OdQGSNFsZwJJUxACWpCIGsCQVMYAlqcjc6gKmYs2aNXnLLbdUlyFJ3Yjd7ZgWI+DHH3+8ugRJ6rlpEcCSNBMZwJJUxACWpCLT4pdwkmaOH//4x4yOjvLss89Wl9JT8+fPZ8WKFcybN2/K7zGAJbVqdHSUhQsXsnr1aiJ2e4HAtJKZPPHEE4yOjnLIIYdM+X1OQUhq1bPPPsuSJUtmTPgCRARLlizZ51G9ASypdTMpfHfp5nsygCWpiAEsqdTylauIiJ49lq9ctdc2n3rqKT71qU91Ve+6dev40Y9+1NV7J/KXcJJKPTz6fc684vaene+684/b6zG7Avjd7373Pp9/3bp1nHPOORxwwAHdlPcT+hbAEbEBeCuwIzMPH7f9fcB7gBeAmzPzwn7VIEmTueiii/j2t7/NkUceycknn8xBBx3E9ddfz3PPPcdpp53Gxz/+cZ555hnOOOMMRkdHeeGFF7jkkkt47LHHePjhhznhhBNYunQpmzdvfkl19HMEfBXwB8Af79oQEScApwI/l5nPRcRBfWxfkiZ16aWXsm3bNrZu3crGjRu54YYbuPPOO8lMTjnlFG677TbGxsZYtmwZN998MwA/+MEPOPDAA7nsssvYvHkzS5cufcl19G0OODNvA56csPk/A5dm5nPNMTv61b4kTcXGjRvZuHEjRx11FEcffTQPPPAADz74IEcccQSbNm3iwx/+MF/72tc48MADe95227+EOxT4txFxR0R8NSKO2d2BEbE2IkYiYmRsbKyrxiom9yVNL5nJxRdfzNatW9m6dSvbt2/nvPPO49BDD2XLli0cccQRfPSjH+UTn/hEz9tu+5dwc4HFwBuAY4DrI+LVOclHM2fmemA9wPDwcFcf3VwxuS9p8C1cuJCnn34agDe96U1ccsklnH322SxYsICHHnqIefPmsXPnThYvXsw555zDokWLuPLKK3/ivb2Ygmg7gEeBG5vAvTMiXgSWAt0NcSVNe8tWrOzp4GbZipV7PWbJkiUcf/zxHH744bz5zW/mHe94B8ceeywACxYs4Oqrr2b79u1ccMEFzJkzh3nz5nH55ZcDsHbtWtasWcOyZcte8i/hYpLBZ89ExGrgpl1XQUTEfwKWZeZvRcShwK3AqslGwOMNDw/nyMhIN+33fATcz/6SZoP777+f1772tdVl9MVuvrfd3iLXz8vQrgV+CVgaEaPAx4ANwIaI2AY8D5y7t/CVpJmqbwGcmWftZtc5/WpTkqYTb0WW1LqZ+B/fbr4nA1hSq+bPn88TTzwxo0J413rA8+fP36f3uRaEpFatWLGC0dFRur2+f1Dt+kSMfWEAS2rVvHnz9ulTI2YypyAkqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCJ9C+CI2BAROyJi2yT7PhQRGRFL+9W+JA26fo6ArwLWTNwYESuBXwG+18e2JWng9S2AM/M24MlJdn0SuBDIfrUtSdNBq3PAEXEq8FBm3t1mu5I0iOa21VBEHAD8NzrTD1M5fi2wFmDVqlV9rEySarQ5Av5p4BDg7oj4LrAC2BIRPzXZwZm5PjOHM3N4aGioxTIlqR2tjYAz8x7goF2vmxAezszH26pBkgZJPy9Duxb4OnBYRIxGxHn9akuSpqO+jYAz86y97F/dr7YlaTrwTjhJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklSkbwEcERsiYkdEbBu37X9GxAMR8c2I+EJELOpX+5I06Po5Ar4KWDNh2ybg8Mz8WeBvgYv72L4kDbS+BXBm3gY8OWHbxszc2bz8a2BFv9qXpEFXOQf8H4A/393OiFgbESMRMTI2NtZiWZLUjpIAjoiPADuBa3Z3TGauz8zhzBweGhpqrzhJasncthuMiN8A3gqclJnZdvuSNChaDeCIWANcCPxiZv6ozbYladD08zK0a4GvA4dFxGhEnAf8AbAQ2BQRWyPiD/vVviQNur6NgDPzrEk2f7pf7UnSdOOdcJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqUjfAjgiNkTEjojYNm7b4ojYFBEPNl9f2a/2JWnQ9XMEfBWwZsK2i4BbM/M1wK3Na0malfoWwJl5G/DkhM2nAp9pnn8GeFu/2pekQdf2HPDBmflI8/xR4ODdHRgRayNiJCJGxsbG2qlOklpU9ku4zEwg97B/fWYOZ+bw0NBQi5VJUjvaDuDHIuJVAM3XHS23L0kDo+0A/iJwbvP8XOBPW25fkgZGPy9Duxb4OnBYRIxGxHnApcDJEfEg8MvNa0maleb268SZedZudp3UrzYlaTrxTjhJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKTCmAI+L4qWyTJE3dVEfAvz/FbTPbnLlERE8ey1euqv5uJBXb42poEXEscBwwFBEfHLfrXwD79bOwgfTiTs684vaenOq684/ryXkkTV97W45yf2BBc9zCcdt/CJzer6IkaTbYYwBn5leBr0bEVZn59y3VJEmzwlQXZH9ZRKwHVo9/T2ae2I+iJGk2mGoAfw74Q+BK4IX+lSNJs8dUA3hnZl7e10okaZaZ6mVofxYR746IV0XE4l2PvlYmSTPcVEfAuz5K/oJx2xJ4dW/LkaTZY0oBnJmH9LsQSZptphTAEfHvJ9uemX/c23IkafaY6hTEMeOezwdOArYABrAkdWmqUxDvG/86IhYBn+1HQZI0W3S7HOUzgPPCkvQSTHUO+M/oXPUAnUV4Xgtc36+iJGk2mOoc8O+Oe74T+PvMHO220Yj4r8C76IT6PcBvZuaz3Z5PkqajKU1BNIvyPEBnRbRXAs9322BELAf+CzCcmYfTGVG/vdvzSdJ0NdVPxDgDuBP4deAM4I6IeCnLUc4FXh4Rc4EDgIdfwrkkaVqa6hTER4BjMnMHQEQMAX8B3LCvDWbmQxHxu8D3gP8HbMzMjROPi4i1wFqAVav89AhJM89Ur4KYsyt8G0/sw3t/QkS8EjiVzlUUy4BXRMQ5E4/LzPWZOZyZw0NDQ900JUkDbaoj4Fsi4svAtc3rM4EvddnmLwPfycwxgIi4kc7HHl3d5fkkaVra22fC/QxwcGZeEBG/Bryx2fV14Jou2/we8IaIOIDOFMRJwEiX55KkaWtv0wjr6Hz+G5l5Y2Z+MDM/CHyh2bfPMvMOOnPHW+hcgjYHWN/NuSRpOtvbFMTBmXnPxI2ZeU9ErO620cz8GPCxbt8vSTPB3kbAi/aw7+U9rEOSZp29BfBIRPzHiRsj4l3AXf0pSZJmh71NQXwA+EJEnM0/Be4wsD9wWh/rkqQZb48BnJmPAcdFxAnA4c3mmzPzK32vTJJmuKmuB7wZ2NznWiRpVul2PWBJ0ktkAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJalISQBHxKKIuCEiHoiI+yPi2Io6JKnS3KJ2fw+4JTNPj4j9gQOK6pCkMq0HcEQcCPwC8BsAmfk88HzbdUhStYopiEOAMeCPIuIbEXFlRLxi4kERsTYiRiJiZGxsrP0qJanPKgJ4LnA0cHlmHgU8A1w08aDMXJ+Zw5k5PDQ01HaNktR3FQE8Coxm5h3N6xvoBLIkzSqtB3BmPgp8PyIOazadBNzXdh2SVK3qKoj3Adc0V0D8HfCbRXVIUpmSAM7MrcBwRduSNCi8E06SihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAq8yZS0T07LF85arq70jSPqpajlIv7uTMK27v2emuO/+4np1LUjscAUtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAS1IRA1iSihjAklTEAJakIgawJBUpC+CI2C8ivhERN1XVIEmVKkfA7wfuL2xfkkqVBHBErAB+Fbiyon1JGgRVI+B1wIXAi0XtS1K51gM4It4K7MjMu/Zy3NqIGImIkbGxsZaqk6T2VIyAjwdOiYjvAp8FToyIqycelJnrM3M4M4eHhobarlGS+q71AM7MizNzRWauBt4OfCUzz2m7Dkmq5nXAklSk9FORM/Mvgb+srEGSqjgClqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIGsCQVMYAlqYgBLElFDGBJKmIAa9pZvnIVEdGzx/KVq6q/Jc1SpctRSt14ePT7nHnF7T0733XnH9ezc0n7whGwJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSkdYDOCJWRsTmiLgvIu6NiPe3XYMkDYKK5Sh3Ah/KzC0RsRC4KyI2ZeZ9BbVIUpnWR8CZ+UhmbmmePw3cDyxvuw5JqlY6BxwRq4GjgDsm2bc2IkYiYmRsbKz12iSp38oCOCIWAJ8HPpCZP5y4PzPXZ+ZwZg4PDQ21X6Ak9VlJAEfEPDrhe01m3lhRgyRVq7gKIoBPA/dn5mVtty9Jg6JiBHw88E7gxIjY2jzeUlCHJJVq/TK0zPwrINpuV5IGjXfCSVIRA1iSihjAklTEAJakIgawJBUxgCWpiAEsSUUMYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAZ4o5c4mInj2Wr1zV0/KWr1zVs9p6rsd9N3f/+QP9Z9FLvfxzHfSfu37UV/GpyOqHF3dy5hW39+x0151/XM/OBfDw6Pd7Vl+va+tH3w3yn0Uv9fLPFQb75w56X58jYEkqYgBLUhEDWJKKGMCSVMQAlqQiBrAkFTGAJamIASxJRQxgSSpiAEtSEQNYkooYwJJUxACWpCIlARwRayLiWxGxPSIuqqhBkqq1HsARsR/wv4E3A68DzoqI17VdhyRVqxgB/zywPTP/LjOfBz4LnFpQhySVisxst8GI04E1mfmu5vU7gX+Tme+dcNxaYG3z8jDgW30ubSnweJ/b6Ja1dcfaumNt3dldbY9n5prJ3jCwn4iRmeuB9W21FxEjmTncVnv7wtq6Y23dsbbudFNbxRTEQ8DKca9XNNskaVapCOC/AV4TEYdExP7A24EvFtQhSaVan4LIzJ0R8V7gy8B+wIbMvLftOibR2nRHF6ytO9bWHWvrzj7X1vov4SRJHd4JJ0lFDGBJKmIAAxHx3Yi4JyK2RsRIcS0bImJHRGwbt21xRGyKiAebr68coNr+e0Q81PTd1oh4S1FtKyNic0TcFxH3RsT7m+3lfbeH2sr7LiLmR8SdEXF3U9vHm+2HRMQdzXIB1zW/MB+U2q6KiO+M67cj265tXI37RcQ3IuKm5vU+9ZsB/E9OyMwjB+Aaw6uAiRdtXwTcmpmvAW5tXle4in9eG8Anm747MjO/1HJNu+wEPpSZrwPeALynucV9EPpud7VBfd89B5yYmT8HHAmsiYg3AL/T1PYzwP8Fzhug2gAuGNdvWwtq2+X9wP3jXu9TvxnAAyYzbwOenLD5VOAzzfPPAG9rs6ZddlPbQMjMRzJzS/P8aTp/KZYzAH23h9rKZcc/NC/nNY8ETgRuaLZX9dvuahsIEbEC+FXgyuZ1sI/9ZgB3JLAxIu5qboEeNAdn5iPN80eBgyuLmcR7I+KbzRRFyfTIeBGxGjgKuIMB67sJtcEA9F3z3+itwA5gE/Bt4KnM3NkcMkrRPxgTa8vMXf32202/fTIiXlZRG7AOuBB4sXm9hH3sNwO4442ZeTSdFdreExG/UF3Q7mTnusGBGQUAlwM/Tee/iI8A/6uymIhYAHwe+EBm/nD8vuq+m6S2gei7zHwhM4+kc1fqzwP/qqKOyUysLSIOBy6mU+MxwGLgw23XFRFvBXZk5l0v5TwGMJCZDzVfdwBfoPNDOEgei4hXATRfdxTX848y87HmL8mLwP+hsO8iYh6dgLsmM29sNg9E301W2yD1XVPPU8Bm4FhgUUTsulGrfLmAcbWtaaZ0MjOfA/6Imn47HjglIr5LZ0XHE4HfYx/7bdYHcES8IiIW7noO/Aqwbc/vat0XgXOb5+cCf1pYy0/YFW6N0yjqu2b+7dPA/Zl52bhd5X23u9oGoe8iYigiFjXPXw6cTGeOejNwenNYVb9NVtsD4/5BDTpzrK33W2ZenJkrMnM1neUUvpKZZ7Ov/ZaZs/oBvBq4u3ncC3ykuJ5r6fx39Md05pDOozO3dCvwIPAXwOIBqu1PgHuAb9IJu1cV1fZGOtML3wS2No+3DELf7aG28r4Dfhb4RlPDNuC3mu2vBu4EtgOfA142QLV9pem3bcDVwIKKn7lxdf4ScFM3/eatyJJUZNZPQUhSFQNYkooYwJJUxACWpCIGsCQVMYA1K0TE2yIiI2Jg7vKSDGDNFmcBf9V8lQaCAawZr1mD4Y10bhx5e7NtTkR8KiIeaNYJ/lJEnN7se31EfLVZnOnLE+5Yk3rGANZscCpwS2b+LfBERLwe+DVgNfA64J101j/YtWbD7wOnZ+brgQ3Ab1cUrZmv9U9FlgqcRWehFOgsnHIWnZ/9z2VnIZxHI2Jzs/8w4HBgU2epAfajc/u11HMGsGa0iFhMZ6WqIyIi6QRq0ln1btK3APdm5rEtlahZzCkIzXSnA3+Smf8yM1dn5krgO3Q+2ePfNXPBB9NZUAXgW8BQRPzjlERE/OuKwjXzGcCa6c7in492Pw/8FJ0V3e6js6LWFuAHmfk8ndD+nYi4m87KZce1Vq1mFVdD06wVEQsy8x8iYgmdJQSPz8xHq+vS7OEcsGazm5oFv/cH/ofhq7Y5ApakIs4BS1IRA1iSihjAklTEAJakIgawJBX5/0R2OfbsYuj3AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "sns.displot(y_train,label='train')\n", "plt.legend()\n", "sns.displot(y_test,label='test')\n", "plt.legend()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Model fit\n", "\n", "Now lets go ahead and fit the model\n", "- we will use a fairly standard regression model called a Support Vector Regressor (SVR)\n", " - similar to the one we used in the previous section" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "SVR(kernel='linear')" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.svm import SVR\n", "\n", "# define the model\n", "lin_svr = SVR(kernel='linear')\n", "\n", "# fit the model\n", "lin_svr.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Model diagnostics" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Now let's look at how the model performs in predicting the data\n", "- we can use the `score` method to calculate the coefficient of determination (or [R-squared](https://en.wikipedia.org/wiki/Coefficient_of_determination)) of the prediction.\n", " - for this we compare the observed data to the predicted data" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "# predict the training data based on the model\n", "y_pred = lin_svr.predict(X_train) \n", "\n", "# caluclate the model accuracy\n", "acc = lin_svr.score(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "accuracy (R2) 0.9998468951478122\n" ] } ], "source": [ "# print results\n", "print('accuracy (R2)', acc)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now lets plot the predicted values" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 0, 'Predicted Age')" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEHCAYAAACzy817AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAt4UlEQVR4nO3deXxU1d3H8c9vJpnshIQ1bCaIgAIRfXBfilitYK1r7UO1LrVFq1SLFUWtAqFWbGqxra0Vd/vYqhWsigyKCuJStYDsi6gJm4EASSD7MnOeP2agiEACWSaTfN+vly8md+6d+V0vfDmce8655pxDRESilyfSBYiISOMoyEVEopyCXEQkyinIRUSinIJcRCTKKchFRKJcTH07mFk8sACIC+//knNuopk9DXwL2Bne9Rrn3JKDfVbnzp1dZmZmY+oVEWl3Fi1atN051+VA79cb5EA1MMI5V2ZmscD7ZuYPvzfeOfdSQ4vJzMxk4cKFDd1dREQAM1t/sPfrDXIXmjFUFv4xNvyfZhGJiLQSDeojNzOvmS0BCoG5zrmPw2/dZ2bLzGyamcU1V5EiInJgDQpy51zAOTcU6AWcaGaDgTuBgcAJQDpwx/6ONbMxZrbQzBZu27ataaoWEZE9DmnUinOuBJgHnOecK3Ah1cBTwIkHOGa6c26Yc25Yly4H7KsXEZHDVG+Qm1kXM+sYfp0AnAOsMbOM8DYDLgJWNF+ZIiJyIA0ZtZIBPGNmXkLB/6JzbpaZvWNmXQADlgA3NF+ZIiJyIA0ZtbIMOG4/20c0S0UiIm2I3+8nNzeXvLw8srKyGD9+PCNHjmzS79DMThGRZuL3+7lpwmTysy6gwxGDKCgoYOzYsfj9/voPPgQKchGRZlAXCHL33+YTPHcCwS79KRt0EYlJyfh8PnJzc5v0uxrSRy4iIvXYuwulx6AT8Z58FUV9zgTAU1lC0rq5GI7ExETy8/Ob9LsV5CIijeT3+xk7diyx8YnEn/B9NmeeHpoP74J48z4kdf27eAI1AFRUVNDUa04pyEVEGik3Nxdv9/7UHv8DAkmh+TJWWkjiypep2riSSp+PxMREKioqqKmpYfz48U36/eojFxFphLLqOr5MG0bN6TeFQjwYJCHvXdI++SuVG1bw8MMPk5GRQXFxMRkZGTz88MNNPmpFLXIRkcM0f20hd728HI76FgDeXV+RsuoVYsq2UF5eTmZmJiNHjmzy4N6XglxE5BAVl9cw5fVVzFy8GYBYD9QteYXY/A/wJsRT3kxdKAeiIBcRaSDnHLOXb+HeV1awozx08/LEzHSmXjqEtQs95OZ+Tn5+PpmZmc0y8edAFOQiIg2wdVcVv/rXCuau2gpAclwME0YO5Icn9sHjMfq2QBfKgSjIRUQOwjnHC//ZyH2zV1NaVQfAiIFdue/iwWSkJkS4uhAFuYjIAazfUc6dM5fz4Rc7AEhP8jHpe4O4IDuD0MKvrYOCXERkH4Gg46kP8vjdm2upqg0CcOHQHky8YBDpSb4IV/dNCnIRkb2s3VLK7TOWsXRjCQAZqfHcd/FgRgzsFtnCDkJBLiICVNcF+Mu8L/jL/M+pDYSeL3/FSX2YMHIgKfGxEa7u4BTkItLuLd5QzB0vLWNdYRkAWZ2TmHrJEE7q2ynClTWMglxE2q2Kmjp+98ZnPPVhHs6B12P85Iwsxn27P/Gx3kiX12AKchFpl95ft50JM5exqbgSgGMyOvDby7IZ3DM1wpUdOgW5iLQLOTk5TJs2jbKaIF3OGYNv4HAAfDEebjn7KMac2ZdYb3SuI6ggF5E2LycnhylTppDY/xS6jxiDNykNgK6eMv5+8/n065oc4Qobx5xzLfZlw4YNcwsXLmyx7xMRAUjvkUnsKVeScNQpALiaKna9/zdYt4Di4qIIV1c/M1vknBt2oPfVIheRNss5x0uLNpH8/al4ElIAqFm/hNJ50wns2kZtbW2EK2waCnIRaZM2FlVw18vLeW/ddjwJKQSrSil77xmq1ywAIBAIkJwc3V0quynIRaRNCQQdz/47n9w31lJREwAgy1vEh0+Ng6pSvF4vgUCAYDDIuHHjIlxt06g3yM0sHlgAxIX3f8k5N9HMsoDngU7AIuBHzrma5ixWRORg1m0t5Y4Zy1i8oQSArilxTLloMN8Z1J2cuLzQqJWyMpKTkxk3bhz33ntvZAtuIvXe7LTQEl9JzrkyM4sF3gduAW4FZjrnnjezvwJLnXOPHOyzdLNTRJqa3+/nt797kLz4fjBoJHhC7dPRJ/ZmwsijSU1o3dPrG6K+m531Dpp0IWXhH2PD/zlgBPBSePszwEWNK1VE5ND4/X5uuucB8vv/AIZcEArxsu3ckm3cf0l2mwjxhmjQ6Hcz85rZEqAQmAt8AZQ45+rCu2wCeh7g2DFmttDMFm7btq0JShYRgcqaABOe/5jg2b/EpfYAFyRh/QfEz3+Q1x7/XaTLa1ENutnpnAsAQ82sI/AyMLChX+Ccmw5Mh1DXymHUKCLyNf/+YgcTZixjZ8YJAHjLtpK86hVid23GxcWSn58f2QJb2CGNWnHOlZjZPOAUoKOZxYRb5b2Azc1RoIjIbruqarl/9hr+8cmG0IZggJi1b5H61ceYC41QqaioIDMzM3JFRkC9XStm1iXcEsfMEoBzgNXAPOCy8G5XA680U40iIsxdtZVzfv/unhA/rk9H7jkxBrfidSrKduGco7y8nJqaGsaPHx/haltWQ1rkGcAzZuYlFPwvOudmmdkq4Hkz+zXwKfBEM9YpIu2M3+8nNzeXL7/aRtIZ11DeKdSjmxDrZfx3BnD1qZl4PUaPJCM3N5f8/HwyMzMZP358xJ5mHylaa0VEWh2/389NY8fi7XsydUMuxPmSABjYER4bcxa90xMjW2AL01orIhJ17v/DX+HMn1Hb7WgArLaSmOWvUF23md4Tzo9wda2PglxEWo1g0PHcx+vZMOgqiI0HwFe4iuQ1r2PVpawvLo5wha2TglxEWoUvtpUxYcYy/pNfHArxql2kfDabuMLVAJS3w9EoDaUgF5EWtfsmZl5eHllZWdz6y9vYkNSfh95aR01dEIBTusHHj/6GOqvDl5hIRUVFuxyN0lAKchFpMX6/n7Fjx+Lz+UhPT2dzhTH21Y2QFhp00SstgfsvGcIZR3XBP9DT7kejNJRGrYhIixkxYgQFBQUkpqRSkTWcyiNOA48HnOPa07O47dwBJMWpfbkvjVoRkVYjLy+PlL7HUXLMhQSSOgPgLSsk+PHfmPjAOxGuLnopyEWkRZRW1ZIy/Cfs6jY0tCEYICH/PdxKPz26dY1obdFOQS4ize6dNVu5++UVe0LcijfSYc2r1BTmUaubmI2mIBeRZlNUXkPOayv515KvAIiP9TCqt+PT5//J+o15uonZRBTkItLknHO8tqyASa+upKg89ATIk/umM/WSbDI7J8GYURGusG1RkItIk8jJyWHatGlUOB+dR44l9ojjAUiJi+HOUUcz+sTehJ4cKU1NQS4ijZaTk8OUKb8m+djv0O3Mq/D4Qota9fGW8OKtl9I9NT7CFbZtGkcuIo3W6YiB+E6/lrjegwEIVuxk57zH8GxaQrHWR2k0jSMXkWZTFwjyxPt5JF/2Gyw2DoCqNe9S9t4zBCtLqa2tjXCF7YOCXEQOy6qvdnHHjGUs37wTi40jsGsbZfMfo2b9EgACgQDJycmRLbKdUJCLyEHtu8jVL355G+tij+SR+V9QFwx1zR4dU8jbT9+MBWrwer0EAgGCwSDjxo2LcPXtg4JcRA5o30WuNlX5uPn1AugQCvC+nZOYemk2J2alk+PbyLRp0ygrKyM5OZlx48Zx7733RvgM2gfd7BSRA9qzyFWHNMqPPJuq3ieBGbggN551FDeffRTxsd5Il9nm6WaniBy2vLw8ko86keKjLyCYkAaAd9dXBD96ltsfeC/C1cluCnIR2a/i8hqSzv4ZpZ0HhTYE6kj8ch7B1XPp0b1bZIuTr1GQi8jXOOeYvXwL9766grJwiHu2f0mHta9RvX2jFrlqhRTkIrLH1l1V3POvFby5aisAST4vFxwR5ONPXmb9Zj2pp7WqN8jNrDfwLNANcMB059wfzGwS8FNgW3jXu5xzs5urUBFpPs45Xly4kV+/vprSqjoAhg/own0XD6FnxwS4TotctWYNaZHXAb90zi02sxRgkZnNDb83zTn3u+YrT0Say55FrjyJdBl1CzE9Q90oaYmxTLxgEBcO7aFFrqJEvUHunCsACsKvS81sNdCzuQsTkeaTk5PDlF/fR8rx36X7aVfsmV6f5S3in7deTufkuAhXKIfikMaRm1kmsAAYDNwKXAPsAhYSarUfdHUcjSMXaR069R1C3Bk/xpfRH4BAWRE7334U75aVWuSqFapvHLnnED4oGZgB/MI5twt4BDgSGEqoxf7gAY4bY2YLzWzhtm3b9reLiLSQmrogD731GcmXTtkT4pUr5lL83Djq1i+mrKwswhXK4WjQqBUziyUU4s8552YCOOe27vX+Y8Cs/R3rnJsOTIdQi7yxBYvI4VmysYQ7XlrG2q2lmDeWuuKvKJs3ndrNqwAtchXNGjJqxYAngNXOud/vtT0j3H8OcDGwonlKFJHGqKip48E3P+OpD/IIOvAYDPJuYc6zv8ATrNMiV21AQ1rkpwE/Apab2ZLwtruA0WY2lNCQxHzg+maoT0Qa4YPPtzNh5jI2FlUCMLB7Cr+9LJvsXh3J8W3WIldthBbNEmmDdlbW8pvXV/PCwo0A+Lwebj67H9d/60hivQ2+NSathBbNEmln5qzYwr2vrKCwtBqA/zkijQcuzaZfV/V/t1UKcpE2orC0ikmvrmT28i0AJPq83P6dAVx1SiYejyb2tGUKcpEo55xjxuLNTJm1ip2VoWdkntm/C7+5eDC90hIjXJ20BAW5SBTbWFTBXS8v57112wHomBDLPd89hkuO76np9e2IglwkSuz97MzMrL6c8MNfMmuDh4qaAADnD8lg0vcG0SVF0+vbGwW5SBTw+/2MHj2a0tJSPGk9qOo5krzPHRCga0ocUy4azHcGdY90mRIhCnKRKHDjjTeys7SM1JO+T+qp/4vFxIbe+OID5j47mdSE2MgWKBGlIBeJAgXVPjKufghf1ywA6koK2DHnYWo2Lif1n7+JcHUSaQpykVassibAQ299RrcrczGPFxcMULboNXZ+8ByB6spIlyethIJcpJX69xc7uHPmMvJ3VGAeLzXb8iny/5HarZ+ze0Z2UlJShKuU1kBBLhJhe49GycrKYuy421hY15u/f7wBgFivMdizmdf+9kuCdTU45zAzvF4vt99+e4Srl9ZAQS4SQX6/n7Fjx+Lz+UhPT2dTMI1b5xZDYqjFPbR3Rx64NJsB3VPIiS3QIleyX1o0SySCRowYwfLly9lZFSD1rOtIOmY4ABao5VcXHss1p2bi1fT6dk+LZom0Yp9++im1PYbS7ewxeBM7AFCZv4TaD5/hutzPIlydRAsFuUiEfFVSSdw5vyC1b6ihFawqo2TeE5Qum0tMjP5oSsPpd4tICwsGHc99soEH/GuID4d4xWcfUjT3rwTLi8P7BCNZokQZBblIC/pyWxkTZiznk/wiAIIVJRTP/SsVn324ZzSKx+MhJSUlwpVKNNGjQkSaSU5ODmlpacTExJCW3onL7nmU8/7w3p4Qv+x/evGjTvlUff4RMTExxMfHExMTg5np2ZlySNQiF2kGOTk5TJ48GeccsV2zSDjvZhbW9gKC9OyYwP2XDOHM/l2AY4mzgIYVSqNo+KFIM0hKSqKiupaOp42mw0mXhqbXuyCVS+eQ98ofSIpTG0oaTsMPRSIgkJZJj5E/J7ZTbwBqd2xkh/8P1Hy1lqS4P0e4OmlrFOQiTaisuo7fzllD9yt/C4AL1LHr4xns+ugFgrU1emqPNAsFuUgTmbe2kLtnLuernVUAVBeso2jOH6ndlr9nn8zMzMgUJ22aglykkYrLa5gyaxUzP90MQFyMh1F9HM889mvqSopxzuHxeOjYsSN//rO6VaTp1RvkZtYbeBboBjhgunPuD2aWDrwAZAL5wOXOueLmK1WkdXHOMWtZAZNeXcmO8hoATspK54FLs8nsnMS5vf9Gbm4u+fn5ZGZmMn78eEaOHBnhqqUtqnfUipllABnOucVmlgIsAi4CrgGKnHNTzWwCkOacu+Ngn6VRK9IW+P1+7n/oETZ0PQV6ZgOQEhfDhFEDGX1CHzxa5EqaWH2jVuqdEOScK3DOLQ6/LgVWAz2BC4Fnwrs9QyjcRdq02bNnc+ODz7Fh8FV7QpyvVnDncQGuOOkIhbhExCH1kZtZJnAc8DHQzTlXEH5rC6Gul/0dMwYYA9CnT5/DLlQk0vK3l3PrrPW4YaMBsJpyktfOpvbLj3l8UwY/vGhUhCuU9qrBQW5mycAM4BfOuV17D6Nyzjkz228fjXNuOjAdQl0rjStXpOXVBYI89UE+D85dS1WHUGMkrmApSZ/NwVNbgS8xkfz8/MgWKe1ag4LczGIJhfhzzrmZ4c1bzSzDOVcQ7kcvbK4iRSJlzZZd3PHSMpZu2gmAt7oU7+IXSCnfuGefiooKDSuUiKq3j9xCTe8ngNXOud/v9darwNXh11cDrzR9eSKRUV0X4PdvruW7f3x/T4j/6OQjyB3RgcCmZZSXl+Oco7y8nJqaGsaPHx/hiqU9a0iL/DTgR8ByM1sS3nYXMBV40cyuA9YDlzdLhSItJCcnh2nTplGd3INOo27Bm9YTgL6dk7j/kiGc1LcTMJiEGNOwQmlVtGiWCKEQ//XUXFJPv5LE40Zh5sEFAxzrK+TFidcSH+uNdInSjtU3/FBBLgJ0GXIGcadfS0xqaPBV7bY8St78M76yLRQXa56bRJZWPxQ5iJKKGn79+mqSzp8AgKurofyTf1L56SxcoI6y2toIVyhSPwW5tFv+5QXc88pKtpdVA1CzeRVl7zxKoCQ0PSIQCJCcnBzJEkUaREEubZ7f7yc3N5e8vDyysrIYc/NtzCvtyhsrtwKQ5POS7fJ46fm78XgMr9dLIBAgGAzqkWsSFRTk0qb5/X7Gjh2Lz+cjLT2djbG9mLCgHHyhEB8+oAv3XTyEnh3P4+jYbXrkmkQl3eyUNm3EiBEUFBQQ37kXZQMvoLbTkQB4aiv5/ZWncOHQHnrYg7R6utkp7dqX+fkkHjuK4n5ng9cHgG/LMmo//gcXPbgswtWJNA0FubRZn20tJW7UnVQk9wDAU7WLpDWzqFu/mF4ZGRGuTqTpKMilzampC/KX+Z/z53mfUxsOcW/+v+mQ9w5Vu4o1pV7aHAW5tClLNpZwx0vLWLu1FIAjOiVycc9KXv3kQ/K3bdGUemmTFOTSJlTU1PH7Nz/jyQ/yCDrwGPzkjL6M+3Z/EnxefvFDrRUubZeCXKLeh59vZ8LM5WwoqgBgYPcUHrg0m2N7d4xsYSItREEuUWtnZS33z17N8/8JrQ3u83oYO6IfN3zrSHwx9a7QLNJmKMglKr2xcgv3/GsFhaWh6fXH9+nIA5dmc1S3lAhXJtLyFOQSVbaVVjPp1ZW8vjy0HkpCrJfbzxvAVadk4tWDj6WdUpBLVHDOMXPxZnJmrWJnZWhFwjOO6sxvLh5C7/TECFcnElkKcmn1NhVXcNfLK1jw2TYAUhNi+dX5R3PZ//TS9HoRFOTSigWDjmf/nc9v31hLRU0AgFFDujPpe4PomhIf4epEWg8FubRKnxeWcseM5SxaH3o6T5eUOKZcOIjzBmtqvci+FOTSqtQGgjz67hf88e3PqQkEAfjBsN7cNepoUhNjI1ydSOukIJdWY/mmndw+YxmrC3YB0Ds9gfsvzub0ozpHuDKR1k1BLhFXVRtg2luf8fh7eQSCDo/Bj0/L4tZz+5Po029RkfroT4lE1Edf7uDOmcvJ214OQP9uyTxwaTbH9UmLcGUi0aPeIDezJ4HvAoXOucHhbZOAnwLbwrvd5Zyb3VxFSttTWlXLVP8anvt4AwCxXuPG4f246ax+ml4vcoga0iJ/GngYeHaf7dOcc79r8oqkzXt79VZ+9a8VFOysAuDYXqk8cFk2A7t3iHBlItGp3iB3zi0ws8wWqEXauB1l1Ux+bRWvLv0KgPhYD7edO4BrT8vS9HqRRmhMH/lYM7sKWAj80jlXvL+dzGwMMAagT58+jfg6iVbOOV5d+hWTXl1JcUVoev2pR3bi/kuGcESnpAhXJxL9zDlX/06hFvmsvfrIuwHbAQdMATKccz+u73OGDRvmFi5c2KiCJboU7Kzk7pdX8M6aQgBS4mP41flHc/mw3ppeL9JAZrbIOTfsQO8fVovcObd1ry94DJh1OJ8jbVcw6Pj7JxuY6l9DWXUdAOce040pFw2mWwdNrxdpSocV5GaW4ZwrCP94MbCi6UqSaPfltjImzFzOJ3lFAHRO9jH5e4MZNaS7WuEizaAhww//AQwHOpvZJmAiMNzMhhLqWskHrm++EiVa1AWCPP5+HtPmfkZ1XWh6/SXH9+Se848hLckX4epE2q6GjFoZvZ/NTzRDLRLFVn61kztmLGPF5tD0+p4dE/jNJUP4Vv8uEa5MpO3TzE5plKraAH96Zx2PvvsldUGHGVx9Sia3fWcAyXH67SXSEvQnTQ7bwvwibp+xjC+3habXH9kliQcuzWZYZnqEKxNpXxTkcsjKquvInbOGZz9aj3MQ4zF+NvxIbjqrH/Gx3kiXJ9LuKMjlkMxfW8jdL69gc0klAEN6pvLApdkc00PT60UiRUEuDVJcXsOU11cxc/FmAOJiPNx6Tn+uOz2LGK8WuRKJJAW5HJRzjtnLtzDx1RVsL6sB4MSsdB64NJuszppeL9IaKMjlgLbuquKef63gzVWhibzJcTHcOWogo0/og0eLXIm0Ggpy+QbnHC/8ZyP3zV5NaVVoev2IgV257+LBZKQmRLg6EdmXgryd8/v95ObmkpeXR1ZWFteMvY05O9L58IsdAKQn+Zj0vUFckJ2h6fUirZSCvB3z+/2MHTsWn89HWnonNiQexb0fVkNMKMQvHNqDiRcMIl3T60VaNQV5O5abm4vP5yOuaxa7jrmQutReAHirS3ns+rMYMbBbhCsUkYbQuLF2yO/3M2LECBZ88CE7epxMyUk37AnxuI2fEHw9RyEuEkXUIm9ndnenxHTrR/erHyImvXfojdJCUtfOombzKrJ6ZUS2SBE5JAryduaBBx+C4y+juu8ZxJjhggHKF71C9aKXScjoRk1NDePHj490mSJyCBTk7ch767aRP/CHkNwZAG9pAfbJc1SvWUhVVRUZGUMZP348I0eOjHClInIoFORtnN/vZ+rv/8j6tOOh76mhEA/Ukpj3LgnrP8BckNhevcjIyOCdd96JdLkichgU5G3Q7rHhq1atoqrzQFLPHoMlpAIQ3LqO2g+fJsFTDYmJlFdUqDtFJMopyNsYv9/PtddeS1nAQ+LpP6bjgNNCb9RWkfTFWwTWvktcfCydOmWQn59PZmamulNEopyCvA3Ye3ZmYWEh3n6nkX7Wj/HEpwBQ+eVCyuY/Tuc+3XCJCRQXF7N06dIIVy0iTUVBHuX2np2Z2qMvFSdeR0LWcQAEKndR/PbjlK98JzS9vk83KioqyMzMjGzRItKkFORRancr/P333ycQdKQc/11ST76ShNh4ACpWL6DsvaepKtm255jy8nL1h4u0QQryKOP3+5kwYQIrVqwgGAwS26k3XUfeQlzPgQDUle6g6M2/UPn5xyQmJhITE0NtbS0xMTFkZGSoP1ykDVKQR5GcnBzuv/9+qqqqwBND6qmXk3rqDzBvLABlS+dQ+v7fqCnbicfjoba2ltjYWDp16sSTTz6pABdpoxTkUcLv93P//fcTDAbxdT+KTiNvxtc1C4Da4gKK5vyJqg3LiIuLw+PxMHjwYEpLSzUqRaQdqDfIzexJ4LtAoXNucHhbOvACkAnkA5c754qbr8z2Kycnh2nTplFSUoLFxJH2ratIPv4CzOPFBQPsWvgKO997DldXDYDX62XAgAEalSLSjjRk9cOngfP22TYBeNs5dxTwdvhnaUI5OTkkJCQwceJESkpKiOszhIwf/4mUYRdhHi81hXls+dttlMx7ck+Ix8XF0b17d6ZOnRrh6kWkJdXbInfOLTCzzH02XwgMD79+BpgP3NGUhbVnV155Jc899xwAFpdE2vBrSRka+rvUBWrZ+eEL7PzoJQiGHsNmZpgZAwYMYOrUqepGEWlnDrePvJtzriD8egtwwMWrzWwMMAagT58+h/l17cPuESnLli0DIKHfiaSfexMxKZ0AqN68hh1z/kjt9g14vV6cx0NKSgrHH3+8+sFF2rFG3+x0zjkzcwd5fzowHWDYsGEH3K+92z2xZ9OmTXgSU0n/9vUkHX0mAMGaKkree5bSRbMwHB6PhzPPPFPhLSLA4Qf5VjPLcM4VmFkGUNiURbVHubm51NTUEHvUaXQ9+6d4EzoAUJn/KUVzHqZu51YAvDEx3HPPPdx7772RLFdEWpHDDfJXgauBqeFfX2myitqpL7eWEDj9BjofMRSAQFUZxW8/RvmKt7+2n0JcRPbVkOGH/yB0Y7OzmW0CJhIK8BfN7DpgPXB5cxbZlgWDjv/7eD026h583tDT6svXfkDR3EcIlpfs2S85OZkTTjhBIS4i39CQUSujD/DW2U1cS5u3e0x4aWkpcXFxdO2XjRv2v9ClH3h9BMqL2fXOY1R89iHBuro9x8XFxdG1a1etkSIi+6WZnS1k7yGFeLzEZJ9P8LQfYjGh6fWW9xGn+TbxypefABAfH49zjkAgoGGFInJQCvIW4Pf794S4r9uRoen13Y4EoK5kK+lfzqF243IKMzKYOXMmubm5euiDiDSYgryZ7P2wh9D0eh+pp46mw0mXhKbXuyCli2ZRsuBZumcPIjYxkfz8fEaOHKngFpFDoiBvBrtXKQwEAsTFxVGXlknGZWOJ7dQLgJrt69nh/xM1X63Zc4we+CAih0tB3sSys7NZvnw5AOZLIOWUH9Hp+PMBcIE6dn70Ijv//SIE/nszUw98EJHGUJA3kZycHCZPnkwwGAQgoe8w0r9zEzEdugBQXfAZO/x/pHZb/teO69ixox74ICKNoiBvAjk5OUyZMoVgMIgnoQNpZ/+U5EFnARCsraLkveeo+HQWMV4PHo+HYDBIUlISt99+u8aFi0ijKcibwLRp03DOkXj0maR/+3q8iakAVK1fyo45f6KuZAvx8fHMnDlTrW4RaXIK8sO096iU0oCX9IvuJrHfiQAEq8spnvckZUvf2LP/nXfeqRAXkWZhzrXcgoTDhg1zCxcubLHvaw67l5pdtWoVPl8c3U69hLrBF+CJSwSgYt1HFL35CIGyHXuOMbM9feciIofKzBY554Yd6H21yA+B3+/nsssuo6Kigpi0HqSc93OCfYbgAQLlJRS99Vcq1rz/jeMmTZrU4rWKSPuhID8E11xzDRWVVXQ48RJST78CT2wcAGXL36Zk3hMEKnd9bf8YLTkrIi1AQX4Iil0S3X90J3EZRwFQt7OQHW88TFXeYgASExMZOHAgFRUV1NTU8PDDD6tfXESanYK8AarrAjz8zudkXD0N84b+l+1a9BolC57F1VQCoafX9+vXj+LiYq2RIiItSkFej0Xri7hjxnI+LyzDvDHU7tjIDv+fqN686mv7DRo0iKVLl0aoShFpzxTkB1BeXUfuG2t55t/5OAdej5FRsoIPnvoVBGq/tm9iYiJTp06NUKUi0t55Il1Aa5CTk0NaWhoxMTGkpaUxZuJDnDttAU9/GArxwT078OrY03j/L7dzxf9ejsfz3/9tXbt25aWXXlI3iohETLseR757lcKqqioAYhJTSTv7JyQeE5peHxfj4Rff7s9Pz8gixqu/80QkMjSO/AB2r49SF36kWuKA00g/5wa8SWkA1H21mncevIG+XZIjWaaISL3aXZDvnlr/7rvvEgwG8Sank37Oz0jsfwoAweoKiuc/RdmSOfR99rYIVysiUr92FeT7LjWbnH0OaWddhyc+1Oqu+PwTit78C4HS7ZhZJEsVEWmwdhPkZ511FvPnzwcgJrUb6efdTELmsQAEKnZS9NajVKxeEMEKRUQOT5sPcr/fzzXXXENhYSGYh5T/uYCOZ/4IT2w8AOUr51P09nSCe02v93g8dOjQIVIli4gckkYFuZnlA6VAAKg72F3VSNh7VEps5yPoNPJm4noMAKCudDtFb/yZyi/+843jPB4P48aNa+lyRUQOS1O0yM9yzm1vgs9pMn6/n5tuuom8vDzwxJB62mhST7kc88YCUPqpn+L5T+6ZXr+3+Ph47rzzTi10JSJRo011rexeK3zFihUEg0F8Gf3pNPIWfF2OAKC2aDM75vyJ6o0rvnFsx44dGTdunAJcRKJOY4PcAW+amQMedc5N33cHMxsDjAHo06dPI7/uwPx+P2PHjmXLli0Q4yPt9CtJGfY9zDy4YIBdn7zMzg/+jqur+caxV1xxBf/3f//XbLWJiDSnxgb56c65zWbWFZhrZmucc18b+hEO9+kQmtnZyO/br90PfKisrCSuTzbdz/s5sR27A1Cz9Ut2+P9AzdYv9nvs8OHDFeIiEtUaNe/cObc5/Gsh8DJwYlMU1VA5OTkkJyczatQoKgNG2nfG0u1/7yO2Y3dcXS3FC56l4Nlx+w3xzMxMZs+ezbx581qyZBGRJnfYLXIzSwI8zrnS8OtzgZwmq6weOTk55OTkEAgESDjqZNLP+RkxKZ0AqNq0ih3+P1JXtOkbxyUlJXH77berL1xE2ozGdK10A14Oz4CMAf7unJvTJFUdgN/v58Ybb2TDhg0Eg0E8iR3pfM71JA08A4BgTSUl7z5D6eLXCXXfhx58HBMTw9FHH83UqVO1SqGItDmHHeTOuS+BY5uwloPy+/2MHj2anTt3ApA0eARpI36KNyEFgMovF7HjjYcJ7Nq255js7Gw97EFE2rxWP/wwJyeHSZMmsXu5XW+HrnQ6bywJWccDEKgspfjtxyhf+c7XjvP5fHrYg4i0C606yHNycpg4cWLoB/OQctwoOn7rajy+BADKVy+g6K3pBCtKvnacz+fj7rvvVjeKiLQLrTrIp02bBkBsp96kj7yZ+J5HA1BXuoOiuY9Que6jbxyTnZ2tvnARaVdadZCXlpYS27kPGVf/AYsJT69f+gbF857EVZd/Y//JkydrNIqItDutOshTUlIo2b6ByvVLiE3vRdEbD1O1fv83L6+44gqFuIi0S636QZS7VyDcMfshCp4a+40QNzMSExOZPHmyZmeKSLvVqlvku1vYe49aARgyZAjLli2LVFkiIq2K7R2QzW3YsGFu4cKFLfZ9IiJtgZktOtjzHlp114qIiNRPQS4iEuUU5CIiUU5BLiIS5RTkIiJRTkEuIhLlWnT4oZltA9a32Bf+V2dgewS+t6npPFoXnUfr0pbP4wjnXJcDHdCiQR4pZrbwYGMwo4XOo3XRebQu7fk81LUiIhLlFOQiIlGuvQT59EgX0ER0Hq2LzqN1abfn0S76yEVE2rL20iIXEWmz2nyQm1m+mS03syVmFjVLL5rZk2ZWaGYr9tqWbmZzzWxd+Ne0SNbYEAc4j0lmtjl8TZaY2ahI1lgfM+ttZvPMbJWZrTSzW8Lbo+p6HOQ8ou16xJvZJ2a2NHwek8Pbs8zsYzP73MxeMDNfpGs9mIOcx9NmlrfX9Rha72e19a4VM8sHhjnnomp8qZmdCZQBzzrnBoe3/RYocs5NNbMJQJpz7o5I1lmfA5zHJKDMOfe7SNbWUGaWAWQ45xabWQqwCLgIuIYouh4HOY/Lia7rYUCSc67MzGKB94FbgFuBmc65583sr8BS59wjkaz1YA5yHjcAs5xzLzX0s9p8izxaOecWAEX7bL4QeCb8+hlCfwhbtQOcR1RxzhU45xaHX5cCq4GeRNn1OMh5RBUXUhb+MTb8nwNGALvDLxqux4HO45C1hyB3wJtmtsjMxkS6mEbq5pwrCL/eAnSLZDGNNNbMloW7Xlp1l8TezCwTOA74mCi+HvucB0TZ9TAzr5ktAQqBucAXQIlzri68yyai4C+pfc/DObf7etwXvh7TzCyuvs9pD0F+unPueGAkcFP4n/pRz4X6xKK1X+wR4EhgKFAAPBjRahrIzJKBGcAvnHO79n4vmq7Hfs4j6q6Hcy7gnBsK9AJOBAZGtqLDs+95mNlg4E5C53MCkA7U213X5oPcObc5/Gsh8DKhix6ttob7OXf3dxZGuJ7D4pzbGv4NHAQeIwquSbgPcwbwnHNuZnhz1F2P/Z1HNF6P3ZxzJcA84BSgo5ntfg5xL2BzpOo6VHudx3nhLjDnnKsGnqIB16NNB7mZJYVv6mBmScC5wIqDH9WqvQpcHX59NfBKBGs5bLvDL+xiWvk1Cd+UegJY7Zz7/V5vRdX1ONB5ROH16GJmHcOvE4BzCPX3zwMuC+8WDddjf+exZq/GgRHq56/3erTpUStm1pdQKxwgBvi7c+6+CJbUYGb2D2A4oZXQtgITgX8BLwJ9CK0ieblzrlXfSDzAeQwn9M94B+QD1+/V19zqmNnpwHvAciAY3nwXof7lqLkeBzmP0UTX9cgmdDPTS6gx+qJzLif85/15Qt0RnwJXhlu1rdJBzuMdoAtgwBLghr1uiu7/s9pykIuItAdtumtFRKQ9UJCLiEQ5BbmISJRTkIuIRDkFuYhIlFOQS8SZWSC8ytsKM/unmSU24rOeNrPLwq8fN7NjDrLvcDM79TC+I9/MOh/gvaFm5szsvEP9XJHDpSCX1qDSOTc0vDpiDaHV3/bYa7beIXHO/cQ5t+oguwwHDjnI6zGa0Cp2o5v4c0UOSEEurc17QL9wa/k9M3sVWBVeXCjXzP4TXkzoegjNfjOzh81srZm9BXTd/UFmNt/MhoVfn2dmi8NrP78dXjTqBmBc+F8DZ4Rn2s0If8d/zOy08LGdzOxNC60Z/TihiRrfEJ6J931Cy9ueY2bxe713T7jG983sH2Z2W3j7kWY2J7yo23tmFpVrhkhkHVZLR6Q5hFveI4E54U3HA4Odc3nhlSt3OudOCK8G94GZvUloBb8BwDGEVh9cBTy5z+d2IbSGyJnhz0p3zhVZaM3qPetwm9nfgWnOuffNrA/wBnA0odmo74dn3Z0PXHeAUzgVyHPOfWFm84HzgRlmdgJwKXAsoaVKFxNaCxxCz2e8wTm3zsxOAv5CaDlWkQZTkEtrkBBeyhNCLfInCIXiJ865vPD2c4Hs3f3fQCpwFHAm8A/nXAD4Kjy9eV8nAwt2f9ZBptF/Gzgm1LAGoEN4pcAzgUvCx75uZsUHOH40oSnihH+9itACVacBrzjnqoAqM3sN9qxCeCrwz72+s94lS0X2pSCX1qAyvJTnHuFgK997E/Bz59wb++zXlI8l8wAnhwN331oOysy8hFrdF5rZ3YTq7bR70baDfF/JvucucqjURy7R4g3gZ+FlWDGz/uEVLRcAPwj3oWcAZ+3n2I+AM80sK3xsenh7KbB30L4J/Hz3D/bfZyUuAH4Y3jYS2N+DF84GljnnejvnMp1zRxBqjV8MfABcYKFnNCYD3wUIrwWeZ2bfD3+2mdmxh/I/RQQU5BI9HifU/73YQg9yfpTQvyhfBtaF33sW+Pe+BzrntgFjgJlmthR4IfzWa8DFu292AjcDw8I3U1fx39Ezkwn9RbCSUBfLhv3UN5r/rrS52wxgtHPuP4SWvF0G+AmtPrgzvM8VwHXhulYSenycyCHR6ociLcDMksMP2U0k1MIfs/v5mSKNpT5ykZYxPTw5KR54RiEuTUktchGRKKc+chGRKKcgFxGJcgpyEZEopyAXEYlyCnIRkSinIBcRiXL/D/TdWSQKHPV3AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.regplot(y=y_pred, x=y_train, scatter_kws=dict(color='k'))\n", "plt.xlabel('Predicted Age')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now thats really cool, eh? **Almost a perfect fit**" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "... which means something is wrong\n", "- what are we missing here?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "- **recall**: We are still using the test data sets." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "
\"logo\"
\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Train/test stratification\n", "\n", "Now lets do this again but we'll add some constraints to the predriction\n", "- Well keey the 75/25 ratio between test and train data sets\n", "- But now we will try to keep the characteristics of the data set consistent accross training and test datasets\n", "- For this we will use something called [stratification](https://en.wikipedia.org/wiki/Stratified_sampling)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "# use `AgeGroup` for stratification\n", "age_class2 = info.loc[y_train.index,'AgeGroup']\n", "\n", "# split the data\n", "X_train2, X_test, y_train2, y_test = train_test_split(\n", " X_train, # x\n", " y_train, # y\n", " test_size = 0.25, # 75%/25% split \n", " shuffle = True, # shuffle dataset before splitting\n", " stratify = age_class2, # keep distribution of age class consistent\n", " # betw. train & test sets.\n", " random_state = 0 # same shuffle each time\n", ")" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Let's re-fit the model on the newly computed (and stratified) train data and evaluate it' performace on an (also stratified) test data\n", "- We'll compute again the model accuracy (R-squared) to evalueate the models performance,\n", "- but we'll also have a look at the [mean-absolute-error](https://en.wikipedia.org/wiki/Mean_absolute_error) (MAE), it is measured as the average sum of the absolute diffrences between predictions and actual observations. Unlike other measures, MAE is more robust to outliers, since it doesn't square the deviations (cf. [mean-squared-error](https://en.wikipedia.org/wiki/Mean_squared_error))\n", " - it provides a way to asses \"how far off\" are our predictions from our actual data, while staying on it's referential space" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "from sklearn.metrics import mean_absolute_error\n", "\n", "# fit model just to training data\n", "lin_svr.fit(X_train2, y_train2)\n", "\n", "# predict the *test* data based on the model trained on X_train2\n", "y_pred = lin_svr.predict(X_test) \n", "\n", "# calculate the model accuracy\n", "acc = lin_svr.score(X_test, y_test) \n", "mae = mean_absolute_error(y_true=y_test,y_pred=y_pred)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Lets check the results" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "accuracy (R2) = 0.6593855081680796\n", "MAE = 3.2059201603105882\n" ] } ], "source": [ "# print results\n", "print('accuracy (R2) = ', acc)\n", "print('MAE = ', mae)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "scrolled": true, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 0, 'Predicted Age')" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3EklEQVR4nO3deXzU13no/88zm6QZAZJYhUYYeV+wDUZyEschBGcxjhOvLE6aODf4Or2/0CZNf9y26atuSn73/tKSJr33kqYlxlnaJshLvMQJjh0vke3UtgTGGIzxApgZISSB9tmXc/+Y0SCwJKTRrJrn/XrxQjqa5QyDzjPfc57zHDHGoJRSqvRY8t0BpZRS+aEBQCmlSpQGAKWUKlEaAJRSqkRpAFBKqRJly3cHJmPOnDlm8eLF+e6GUkoVlV27dp0wxsw9s72oAsDixYtpa2vLdzeUUqqoiMh7o7XrFJBSSpUoDQBKKVWiNAAopVSJ0gCglFIlSgOAUkqVKA0ASqkp2blzJ6tWraKhoYFVq1axc+fOfHdJTZAGAKVU2nbu3MnGjRvp6OigpqaGjo4ONm7cqEGgSGgAUEqlbcuWLTgcDlwuFyKCy+XC4XCwZcuWfHdNTYAGAKVU2g4fPozT6Tytzel0cuTIkfx0SE2KBgClVNoaGhrw+/2ntfn9frRkS3HIegAQkXIReUVEXhOR/SLyd8n2BhF5WUTeEZFmEXFkuy9KqczatGkT4XAYn8+HMQafz0c4HGbTpk357pqagFxcAYSAVcaYK4GlwPUi8kHg74HvG2POB3qBDTnoi1Iqg1avXs3WrVupra2lt7eX2tpatm7dyurVq/PdNTUBWS8GZxKHDg8lv7Un/xhgFfC5ZPtPgW8BP8x2f5RSmbV69Wod8ItUTtYARMQqInuALuAp4F2gzxgTTd7EC9Tloi9KKaUSchIAjDExY8xSwA1cDVw80fuKyN0i0iYibd3d3dnqolJKlZycZgEZY/qAZ4EPAVUiMjwF5Qbax7jPNmNMozGmce7c951noJRSKk25yAKaKyJVya8rgE8AB0gEgtuTN7sTeDTbfVFKqWJjjCEUjWXlsXNxIlgt8FMRsZIIOPcbYx4XkTeAHSLy/wGvAttz0BellCoawUiM7sEQrjIbZTZrxh8/F1lAe4Flo7QfIrEeoJRSaoR43NDjDzMQiGT1eYrqTGCllJrufKEoJ4fCROPxrD+XBgCllCoA0Vick74wvlD07DfOEA0ASimVZwPBCD1DYeLG5PR5NQAopVSeRGJxugdDBCPZyfI5Gw0ASimVB/2BCL2+3H/qH0kDgFJK5VA4GuekL0QgnJ9P/SNpAFBKqRyIxw19gQj9gQgmj5/6R9IAoJRSWZbL1M7J0BPBlFIFb+fOnaxatYqGhgZWrVpVNIfOR2NxOgeCdA4EC27wBw0ASqkCt3PnTjZu3EhHRwc1NTV0dHSwcePGgg8C/YEI3t5ATvP6J0sDgFKqoG3ZsgWHw4HL5UJEcLlcOBwOtmzZku+ujSoUjdHeF+DkUCivGT4ToWsASqmCdvjwYWpqak5rczqdHDlyJD8dGkM0FqfXH2EwmN36PZmkVwBKqYLW0NCA3+8/rc3v97N48eL8dOgMxhh6fWG8vYGiGvxBA4BSqsBt2rSJcDiMz+fDGIPP5yMcDrNp06Z8dw1fKIqnJ0CvP3sbulpaWrjllluysgCuAUApVdBWr17N1q1bqa2tpbe3l9raWrZu3ZrXg+jD0TjH+7Of3dPS0sLmzZvp7DyelQVwKZQNCRPR2Nho2tra8t0NpVSJiscNvf4wA8FoTjZzffHOO+nu6sJlFyyRxDSYz+ejtraWZ555ZsKPIyK7jDGNZ7brIrBSSp2FMYaBYJQ+f5hYPHcfmr0eL1VVsyAaTLVlcgFcp4CUUmoc/nAUb28irTOXgz+Au95NIBA4vT8ZXADXAKCUUqOIxBLz/Mf7g0Ri+dnFe9eGDUQiEfz+7CyAZz0AiEi9iDwrIm+IyH4R+Vqy/Vsi0i4ie5J/bsh2X5RS6myMMfQk0zr94fzu4l2xYgX33HMP8+cvyMoCeNYXgUWkFqg1xuwWkRnALuBmYC0wZIz57kQfSxeBlVLZVKhF26qcDmpcjrTvn7dFYGNMB9CR/HpQRA4Addl+XqWUmqhILM7JoXDeP/HnWk7XAERkMbAMeDnZtFFE9orIfSJSPcZ97haRNhFp6+7uzlVXlVIlIB43nBwKFcR0z1gOn/Dxt4/u4w/vnMj4Y+csDVREKoGHgK8bYwZE5IfAtwGT/PsfgS+feT9jzDZgGySmgHLVX6XU9JWvtM6JMsawt72f5lYPLx3qAaAvEOGa8+dk9HlyEgBExE5i8P8PY8wvAYwxnSN+/iPg8Vz0RSlV2oZCUXp94bxl9ownFje8+M4Jmts8HOgYTLWfU+Pk05fXYoxBRDL2fFkPAJLo7XbggDHmeyPaa5PrAwC3APuy3RelVOkKRmKc9IUJRfJ/Fu+ZQpEYT77Ryf1tXtr7TuX9X1o7g/VNi7jhilrmVJZl/HlzcQXwYeALwOsisifZ9k3gDhFZSmIK6AjwlRz0RSlVYqKxOD2+MEMFeDDLYDDCo3uO8fCr7fT6T1US/dC5s1nX5ObyulmICJYMfuofKRdZQC8Ao/X+N9l+bqVU6TLGMBCIZrVSZ7o6B4I8uMvLr1/vIBhJTEXZLMJ1l8xjXVM9i2e7ctIPrQWklJp2AuEYJ4ZCBTfP/273EM2tHp55s4vhtWenw8pnrqjl1qvczJ2R+Wme8WgAUEpNG4U43WOM4VVPH82tHlqP9KbaZ7sc3LbczY1X1FJZlp+hWAOAUqroxeOG/kCEvkAk7TLNLS0t3Lt9O16PF3e9m7s2bGDFihVp9ykWNzz/djc7Wj281TmUaj+nxsnaRjfXXTIfhy2/5dg0ACilitbwPH9fYGr5/MMHr9jtdqqqZtHd1cXmzZu55557Jh0EgpEYv91/nPvbvHT0nyrjfHndTNY21vOh82ZnbVF3sjQAKKWKki8UpSdD+fz3bt+O3W7H6XQCiZr7fr+fe7dvn3AA6A9EeHRPOw+/eoz+QCKjR4Brzp/N+qZ6Lls4a8r9zDQNAEqpopKNuj2pg1dGqKiooN3rPet9O/oDPNDm5Yl9xwlGE8HIbhU+eekC1jS6WVTjzFg/M00DgFKqKBiTmOfv9ac/zz8Wd72b7q6u1BUAQCAQoM7tHvM+b3UO0tzq4fdvdacyelxlVm66ciG3LKtjdhY2bmWaBgClVMELRhJpneFodtI679qwgc2bN+P3+6moqCAQCBCJRLhrw4bTbmeMYdd7vTS3eth1tC/VPreyjNuW13HjFbU4HcUzrBZPT5VSJScai9PjDzMUzG5a5/DBK/du306710ud+/QsoFjc8NzBbppbPbzTfSqjp2GOi3VN9XzsornYrcV3wKIGAKVUwYnHDX2BCP1TSOucrBUrVrxvwTcQibHz9eM8sMtD50Ao1X6lexbrmur5QENNRouz5ZoGAKVUwchUWudU9fnDPPLqMR7Z085A8upDgI9cMId1TfVcUjszb33LJA0ASqmCMBiM0OuL5PU4xva+ZEbP/uOp9Qa7VfjUZQtY2+jGXV24GT3p0ACglMqrTObzjzSZnb1vHh+gudXL82+fyuiZUW7jpqULuXlp3ZTO4y1kGgCUUnkRjsY56QsRCGe+Pv9EdvYaY2g90suOVg97PH2p+86bUcaaRjc3LKmlwmHNeN8KiQYApVROxeOGXn+YgWA0awu84+3svebD1/LMwW7ub/Vw6IQvdZ9z57pY15jI6LEVYUZPOjQAKKVyxh+OcmIwnPV5/tF29pa7ZuK11fFH21+ha/BURs+yRVWsb6qn8Zzqos7oSYcGAKVU1sXihpNDoZyVaR65szduryBUu5Tg/MvBXk7XYAiLwEcvnMu6pnounD8jJ30qRBoAlFJZk6+0zrs2bODvvruVyKJriC+8HCyJoc5mgU9fsZA1y90srKrIWX8KVS4Oha8HfgbMJ3H+7zZjzP8SkRqgGVhM4kzgtcaY3rEeRylVXPzhKCeHMp/dczYHOgZ4um825qNfZTjkWKJBPrKonK/d9CGqnNMzoycdubgCiAJ/bozZLSIzgF0i8hTwJeBpY8x3ROQvgb8E/iIH/VFKZVE0FuekL4wvh6dyxY3h5UM9NLd52OvtT7XXzipnzXI31y9ZQLl9emf0pCMXh8J3AB3JrwdF5ABQB9wErEze7KfAc2gAUKqo9Qci9Ppydwh7JBbn6QNdNLd5eO+kP9V+wbxK1jfVs+LCuVgtpbWwOxk5XQMQkcXAMuBlYH4yOAAcJzFFNNp97gbuBli0aFEOeqmUmqxQNMaJoTChSOZz+kfjC0V5fG8HD+32cmIonGpvPKeadU31XLWoquQyetKRswAgIpXAQ8DXjTEDI98cY4wRkVE/MhhjtgHbABobG/NXHEQp9T7DOf3DJ2Bl24mhEL/c3c6v9h7DF0oEG4vAxy6ax7qmes6fV5mTfkwXOQkAImInMfj/hzHml8nmThGpNcZ0iEgt0JWLviilMmMoFKVnaGI5/VM9cP3oST/3t3l46kAnkVjic2C5zcINV9Ry+3I3C2aWp/06SlkusoAE2A4cMMZ8b8SPHgPuBL6T/PvRbPdFKTV1oWiMHl94wiUcpnLg+r72fna0evjDuydTbVUVdm65qo7PXrmQWRX2Kb2WYuCwWSi3Z2dnci6uAD4MfAF4XUT2JNu+SWLgv19ENgDvAWtz0BelVJrSPZxlsgeux43hP989SXOrh33HBlLtC6vKWdtYz6cunU/ZNM/osVstVJbZcJXZcNiyV5YiF1lAL5AopT2a67L9/EqpqZnq4SwTPXA9HI3zuwOdNLd68PQGUu0XLZjBHU31fPj8OdM6o8duteAqs+Eqs1Jmy02A053ASqlRGWMYCEbp809tF+/ZDlwfCkX51WvHeGh3Oz2+Uxk9VzfUsK7RzdL66ZvRY7NYcJVZcZXZ8rJPQQOAUup9hkJRejNUo3+sA9fXfOHL/Mvv3+XxvR34k+sJVouw6uJ5rGt0c+7c6ZnRY7UIToeNyjJb3stNS67O28yExsZG09bWlu9uKDVtBSMxTvoyn88/nAXU7vUy77wlLLjmVvb1SurKosJu5cYrarntqjrmTcOMHosIzjJrYtC3W3N+RSMiu4wxjWe26xWAUopgJEavf+KZPZP1kY98hKpzr2BHq4eXDvXQeRLAUO20c9tVbj5zZS0zyqdXRo+I4HIkpnecjtwP+hOhAUCpEhaKxujzR7JWtycWN7z47gmaWz0c6BhMtburK1jb6OaTly7IapZLrokIzuFB327FUuCL1hoAlCpB4WicXn/2CraFo3GefOM497d58Y7I6LmkdgbrmxZxzXmzp01Gj4hQbk+mbTpsBT/oj6QBQKkSkm4u/0QNBCI89toxHn61nV7/qfIQHzp3Nuub6llSN7Mgp0ImS0SosFtxlllxOWxFG8w0AChVAqKxOP2BSNbO4e0cCPLgLi+/fr2DYCSROWSzCB+/ZD5rm9wsnu3K+HPmmiU5veMskumdidAAoNQ0Fo3F6QtEGMzSwP9u9xDNrR6eebOL4a0CTsdwRo+buTPKMv6cuWSzWKhwWHGVWfOSvZNtGgCUmoZicUOfP5yVT/zGGPZ4+mhu9fDKkVOH+M12ObjtqjpuvHIhlWXFO7QM78h1OqzT/hCZ4n2XlFLvY4yhz58o25DpQ1liccPzbycyeg52nsroWVTjZF2jm+sumV+0GT3l9sRcfoXDWrSvIR0aAJSaJiZTnnkygpEYv92fyOjp6A+m2pcsnMm6pno+dN5sLEU2NTJdFnGnSgOAUkUuW7t3+wMRHt3TzsOvHksd+CLANefNZl1TPUvqZo3/AAXGapHEfL6jcDdm5ZoGAKWKVCQWp9cXZijDufzH+4M8sMvLztc7CEYTVxN2q/CJS+ezdnk9i2Y7z/IIhcNutaQ2Zk33+fx0aABQqshka4H37c5Bmtu8PHfwVEaPq8zKZ69cyK3L6phdWRwZPWV2K87k9E6uyioXKw0AShUJYwwDgSh9gamVZz7zMXcf7WNHq4dd753K6JlT6eC2q9zceEUtrgLP6Bneiet02HA5rNispbOIO1WF/c4qpQAIhGOcGAplpDwzJK4ifv9WNztaPbzTNZRqXzzbyfqmej528TzsBTyQTsdNWfmgAUCpAhaJxenxZa5mTyASY+frx3lwl5fjA6cyeq5wz2J9Uz0faKgp2MVRm8WCs8yK0zE9N2XlQy4Ohb8PuBHoMsYsSbZ9C/ivQHfyZt80xvwm231RqlgM5/P3pXkM45n6/GEeefUYj+xpZyBZB0iAj1wwh3VN9VxSO3PKz5ENDltiaqcUNmXlQy6uAH4CbAV+dkb7940x383B8ytVVAaDEfr8kYxM97T3BXiwzcvO/ccJj8jouX7JAtYsd+OuLryMnuFNWc4ya0FPQ00HuTgUvkVEFmf7eZQqdr5QlJ4MHcN48PggO1o9PP92dyqjp7LMxk1LF3LLsjpqXI4pP0emDNfQT/wp3U1Z+ZDPNYCNIvJFoA34c2NM72g3EpG7gbsBFi1alMPuKZUbmdrIZYyh9UgvO1o97PH0pdrnzShjTaObG5bU5v0M2mG6Kasw5ORM4OQVwOMj1gDmAycAA3wbqDXGfPlsj6NnAqvpJFOncUVjcZ492E1zq4dDJ3yp9nPnuljfVM/KC+cWRGqkbsrKn4I6E9gY0zn8tYj8CHg8H/1QKh+CkcTA7w9PbeAPhGP8+vUOHtzlpWswlGpftqiKdY31NC2uzvsn6zK7FVdyaqeUiqwVi7wEABGpNcZ0JL+9BdiXj34olUuZOni9xxfm4VfbeXTPsVQZCIvAigvmsq6pnosWzMhEd9Myssia066bsgpdLtJAfwGsBOaIiBf4W2CliCwlMQV0BPhKtvuhVL4EwomBPzjFOX5vr58H2rw8sf84kVhi6tZhs7D6sgWsaXSzsKoiE92dtOFB35WsrKmbsopHLrKA7hileXu2n1epfMvUwH+gY4AdrR5eePsEwyt2M8tt3Ly0jpuXLaTKmZ+MnorkfH4pl1MudmcNAMkF2/8JLDTGrBaRS4EPGWN0EFdqFMFIjB7f1Ab+uDG8criHHa0e9nr7U+0LZpazptHN9UsWUJGHhdQyu5VKhw1XmU7vTAcTuQL4CfBj4K+T378FNKOf4pU6TSbm+COxOM+82UVzq4cjJ/2p9vPnVbK+qZ6PXjg355+27VYLlWU2XGW6kDvdTCQAzDHG3C8ifwVgjImKSGZPnlCqiIWjcXr9U6vX4wtFUxk9J4bCqfbGc6pZ11TPVYuqcprRY7VIatDXlM3payIBwCcis0ks2CIiHwT6x7+LUtNfLG7o8YUZDEbSfoyTQyEe2t3Or147hi955WARWHnRPNY31XP+vMpMdfesRASXw0pluQ2nQ+tEloKJvMvfAB4DzhORF4G5wO1Z7ZVSBcwYQ38gUa8n3YPXj570c/8uD0+90ZnK6Cm3Wbjh8lpuX+5mwazyTHZ5XHarhRnlNmaU23Uxt8ScNQAYY3aLyEeBi0gUEDxojEn/I49SRWwgGKHPF0n74PV97f00t3p48d2TqbZZFXZuXVbHZ5cuZFaFPVNdPStXmY0Z+mm/pE0kC+jWM5ouFJF+4HVjTFd2uqVUYfGHE4XahitqTkbcGP7z3ZM0t3rYd2wg1V47q5y1jW6uv2wBZWnMs7e0tHDv9u14PV7c9W7u2rCBFStWjHufcnsidbOyTFM31QRqAYnIr4EPAc8mm1YCu4AGYLMx5t+y2cGRtBaQyjV/OEqvP5JWobZwNM7TBzppbvNytOdURs9F82ew/up6rj1/TtqDcEtLC5s3b8Zut1NRUUEgECASiXDPPfecFgSGN2klCq9p6mapmkotIBtwyXD9nuS+gJ8BHwBagJwFAKVyZSgUpT+Q3sA/FIry+GvHeGh3Oyd9pzJ6rm6oYX1TPVe6Z005o+fe7dux2+04nYl6/k6nE7/fz73bt7NixQrK7FZmlOsmLTW+iQSA+pHF24CuZFuPiOhagJo2jDEMBKMMBNI7jKV7MMQvd3v51d4O/MmMHqtFWHXxPNY2ujlvbuYyerweL1VVs05rqyhz0H7oIPU1Tj1IRU3IRALAcyLyOPBA8vvbkm0uoC9bHVMqV4YH/n7/2Iu74823Hz7h4/42D08f6CKaPH2l3G7h08mMnvkzM5/R4653093VhbPcgcTCSDSMf2iAhoW1OvirCZvIGoAAtwLXJpt6gfnGmK9muW/vo2sAKpOMMQyGEgP/eJ/4R5tvD0cifOnP/oYD4RpeOtSTum21086tV9Xx2SsXMqM8Oxk9NouFl198jr/4xtcos9tS0z/hcJitW7eyevXqrDyvKl5prwEYY4yIHAI+CKwBDgMPZb6LSuXOZM7dHTnfbhBs7iWEFyzjx28CJAZ/d3UFaxvdfPLSBVkplzB8bOJw2uaiz95ApV3YsmULR44cYfHixWzatEkHfzUpYwYAEbkQuCP55wSJ+j9ijPlYjvqmVMYNhaL0TvLcXa/Hy6zqGkLzLiFYdxXxiurUzy6pncG6pno+fF76GT3jsVstzCy3U1n+/sXc1atX64CvpmS8K4A3geeBG40x7wCIyJ/lpFdKZdjDv/oN3/vBNjztHRPOmYfElcLMq26gv+ZiKDu1iGvpfofZ/QfZ+o2/z3iNHpvFgrPMSqXW4VFZNl4AuBVYDzwrIk8AO0jsBFaqaPhCUR7+zZPc863EHH5V1Sy6u7rYvHnz+3LmR+ocCPLgLi+/fr2DYG1y6jQew9F9EHPoD8T6OvjqPfdkbPC3WSzJGjxWHfRVzowZAIwxjwCPJLN9bgK+DswTkR8CDxtjnsxJD5WapDMXd//lR+PnzI/0bvcQza0ennmzi2RCD06HlaU1MQ4/8wDHj7xFndvNXX86dvCYqOdbWti+7Yccffctzl1Up3P4KufOmgV02o1FqkksBK8zxlyXtV6NQbOA1HjiccNAMEJ/IEIsfur/9apV11FVdfrmK2MM/f39PP300xhj2OPpo7nVwytHelO3me1ycOtVdXzmyoVUlmWmXs7wYu4ffv8Mf/71P6XMYdcsHpV1U9kJnGKM6QW2Jf9M9InvA24EuowxS5JtNSQWlReTOBN4bfKxlZq0WNwwEIgwEDx94B+WyplPXgEABAIBFrrree5gN82tHg52DqZ+tqjGybpGN9ddMj8jGT0WEZzJ83KdDisiwv/5/ncpc9hxuVwAqb+3bNmiAUDlTC52jPwEuP6Mtr8EnjbGXAA8nfxeqUkZrsfv6fHT6w+POvgD3LVhA5FIBL/fjzEGXzBMuPYK/E3/hc2Pv5Ea/C+vm8m3b7qM+77UyOrLa6c0+IsIleU2Fswq55zZTubNKMdVZktdhRw+fPi0gASJqakjR46k/ZxKTVYuDoVvEZHFZzTfRKKoHMBPgeeAv8h2X9T0EI3F6Q9EGAxGJ1SPf8WKFdxzzz3860/+nQ57HZZzP4CxVdAbSmQ1XHP+bNY11rOkbhYtLS186duTq7A5ktUizCy3M7Ni/Nr6DQ0NdHR0pD75A/j9fhYvXjzh51JqqvK1Z3y+MaYj+fVxYP5YNxSRu0WkTUTauru7c9M7VZBC0Rhdg0E8vQH6AxM/jOV4f5C90YX0XvkFuHAlcVsFdqtww+UL+PGXmvj2TUtSg//mzZvp7uo6LVuopaVl1MdtaWnhi3feyapV13HnF/6IPS+1sKjGSbXLcdY9AZs2bSIcDuPz+RJXJT4f4XCYTZs2TfrfRal0TWoROO0nSVwBPD5iDaDPGFM14ue9xpjqMe6eoovApSkQjtEfiOAPT+7M3bc7B2lu8/LcwVMZPa4yK5+9ciG3LqtjdmXZabf/4p13vm+twO/3M3fePH7205+edtvhYOGwGJw2ITDUP+lF3J07d+pOXpUTGVkEzqBOEak1xnSISC2JCqNKnSadkszGGHYf7WNHq4dd753KK5hT6eD25W4+fXktrjEyekatsFlRQbvXe1qb02HjJz/83zjCA1S6nGDSW8TVnbwq3/IVAB4D7gS+k/z70Tz1QxWYiRZoO1Msbvj9W93saPXwTtdQqv2c2U7WN9Wz6uJ5Z62SOVa2UJ3bndqoNaPcht1q4b13D1JTU3Pa/XURVxWbrAcAEfkFiQXfOSLiBf6WxMB/v4hsAN4D1ma7H6qwDefwDwSikzpvNxCJ8cS+4zzQ5uX4QDDVfoV7Fuub6rm6oQbLGbt1xyrtfNeGDWzevBm/35+o+un3EfEP8t+/+lcsmn16xo4u4qrpICdrAJmiawDTTyxu6A9EGJjEoi5Anz/MI68e45E97QwEE2sDAlx7wRzWNdZz6cKZo97vbEcptrS0sP3eH+E99BaL6+bz38eYl9+5cycbN27E4XDoRi5V8MZaA9AAoPJiOJVzIBhlMv8H2/sCPNjm5Yn9xwklD2i3W4XrL1vAmkY37mrnuPcfb6H35//2b8yssDGz3I5lApU9dRFXFYtCWwRWJSoai9OXzOGfzMB/8Pggza0eWt7uTmX0VJbZuGnpQm5ZVkeNyzGhxxl1oddhpf3t/e+b5jkbXcRVxU7PjlM5EYnF6R4M4ekNMBCITGjwN8bQeqSHP3/gNf7bf+zmubeSg7+/j1ntL/Enl0XZcG3DhAd/SCz0BgKBxDexMJZgP6HeThrqF7Jz505WrVpFQ0MDq1atYufOnWm+WqWKg14BqKwKR+P0BcIMBSeewx+NxXkumdFzqNt36gcDndiOvoJr4AhBv4+/3/sUZVaZ1E7d/7phA3/3N98kEOjDVe5Izd2vXLkyNadfU1NDR0cHGzdu1Dl9Na3pFcA0l69PtaFojK6BIN5e/4QH/0A4xkO7vfzR9lf4n795MzX4L62vYs6h3+J45cfMGDiMBYPT6cRut3Pv9u0Temy71cLsyjK+cMtqfvCP/z8L58+lt7eX2tpatm7dynPPPYfD4cDlciEiuFwuHA4HW7ZsSfvf4Ex6haEKjS4CT2P5yFTxhxObtwLhiW/e6vGFefjVdh577RiDyWBhEVhxwVzWNdVz0YIZZy3pPBanw8asCjsVjvEPWWloaKCmpuZ9j9/b28uhQ4cm/FrGollDKp90EbgEbdmyJfWpFrJXctgYk9q1G45OPIff0+PngV1efrv/OJFY4oOIw2ZhdTKjZ2FVReq2423SGk253UqNyzHh07Wyndefq/dCqcnQADCNHT58OKu7VePxxK7dgcDkdu0e6BhgR6uHF94+wfD158xyGzcvrePmZQupcr5/Ufd9m7SS+ft3bdhw2u3sVgs1LseY5R7GsmnTJjZu3Ahw2if0TBVny/Z7oVQ6NABMY9n6VBuMxBgIRvCFYhNO5TTG8PLhHppbPbzm7U+1L5hZzppGN9cvWUDFOJ/Wh0s637t9O+1eb+JYxhGlmm0WC1UuOzPL7Wm9ptWrV7N169as5fXrzmFViHQNYBrL9LxzIByj1x8mOInibJFYnGff7KK5zcvhE6cyes6fV8m6xnpWXjT3rKWTx2O3WphZYWdmuS1jB7Rng64BqHzSncAlaqq7VY0x+JPlmCcz8PtCUX79egcP7vJyYiical9+TjXrGt0sP6d6SgO23WqhymmnsqywB/6RdOewyhcNAGpSorE4g8Eog8HJFWc7ORTil8mMHl8oETAsAisvmse6RjcXzJ8xpX5ZLUKV01Hwn/iVKiSaBaQmJBiJMRCI4AtPfH4f4GiPn/tbPTx1oDOV0VNus7D68lpuX15H7ayKszzC+ESEmeU2qp2OCdXpUUqdnQYAlXY2D8C+9n6aWz384d2TqYyeWRV2blm2kJuW1jGrIr1F2ZHK7VZmVzoos00spbPYDU8VHT58mIaGBp0qUlmjAaDIZHJwCEVjDASi+EITO1x9WNwY/vPdkzS3eth3bCDVXjurnLWN9XzqsvkTzr8fj91qodrloHKSKZ3FbORisZakUNmmawBFJBOZJMObtgaC0UkdtQiJuj5PH+ikuc3L0R5/qv2i+TNY11TPRy6YM6WMnmHDKZ0zimiBN1NWrVr1vnRRn89HbW0tzzzzTB57porZWGsAWguoiIzcTTrZejXRWJweX5ijPX66B0OTGvyHQlF2vHKUz9/7MluefCs1+F+9uJrvrrmCf/78simncwI839LCf/n8OlY0LuHmGz7FE088MaXHG0sh1+Q5fPjwabudQTeMqezRAFBE0hkcwtFTZZj7/GFi8Ylf8XUPhvjX37/L+m0vse35w5z0hbFahCtmw7y3HuaVH/wZ//Q3X+f5559P9yUBiVO6bvr0au763K20vfgcNqslNfWR6cF5+Cqqo6PjtCmWQgkCDQ0N+P3+09p0w5jKlrwGABE5IiKvi8geESnduZ0Jmszg4A9HOd6fqMY5GJxY/f1hR076+IcnDvL5e1+muc2LPxyj3G7htqvq+JMlhn3//m36jx6kqmoW3V1dbN68mZaWltT9W1pa+OKdd7Jq1XV88c47T/vZmf7wQgub/+obHNq/G6tALBbD4/EQjUYzXo0TpnYVlQubNm0iHA7j8/kwxuDz+TJakkKpkQrhCuBjxpilo81PqdOdbXCIxuL0+sIcPenneH8Qf3jiNfiNMez19vHNh1/nyz9p44n9x4nGDdVOOxuuXUzz3R/kqx87nwf//T7sdjtOpxMReV9Z5uEzd7u7usYMEJDI558zo4z7/s93KbMK0WgUq9WK1WpFROjs7MzK1EehT7EMl6Sora09rVy1LgCrbCid9IppYKx6NR/7+CfpGghOOncfEoey/+HdkzS3HuWNjsFUu7u6gjXL3XzqsgU4bKc+J4x6pGJFBe1eLwD3bt+eChBwqrDavdu3p+r2zKywU+10YLVIqkhaWVkZkUgEi8WCxWIhFAplZeqjGGry6FGTKlfyHQAM8KSIGOBfjTHbzryBiNwN3A2waNGinHauEPOxhweH4dz9wWCEY32BST9OOBrnyTc6ub/Ng7f31P0vXjCD9VfX8+HzRs/oOVtZ5vECRIUjUaJ5ZD7/8IA8f/58jh49CiSuRmw2W1amPrJd9VOpYpLvKaBrjTFXAauBr4rI+872M8ZsM8Y0GmMa586dm7OOFepiYTASo3swxNEePyeHQpOqvw8wGIzw85ePcsePXuJ7T72VGvw/eG4N/7TuSn7wuWWsuGDsjJ67NmwgEong9/sTdYL8/tPKMp925m5SwO/jnHlV1M6qeN9mruFpLZvNRn19PRaLhVgsxvnnn5+VqQ+dYlHqlILZByAi3wKGjDHfHes26ewDSPdTfCHlY0dicXyhKEOh6KQH/GFdA0Ee3O3l8b0dBCOJx7BZhOsumcfaxnoa5rjO8gintLS0jFmWeXgNwG63U1FeTnDgJGFfPz8YZ5DVImlKZVfBFYMTERdgMcYMJr9+CthsjBkz+XuyAWAqG6fSPSIwU9NGw1M8Q6HJb9ga6VD3EPe3eXn6za5UCqjTYeXTl9dy+3I3c2eUpf3YY2lpaeG+bf+M5503aThnUckN6IU4dahKWyEGgHOBh5Pf2oCfG2P+x3j3mWwAmMqn+HTum4mduqFoovTyZA5bOZMxhp/9uoXmVg/BmfWp9hqXg9uuquMzVyyksjw7yz+jzfOXEq37rwpRwQWAdEw2AEzloO90fpGnEnAC4Rh9gfCkDlM/UyxueOGdE/zomTc4dursFcR3Eg79gb/+0mdYtfKjaT/+eNI9inG6KaSpQ6WGlWQ56Kmk/KVzROBkz32NxuIMhRI19ydbhXOkUCTGb9/o5IE2L+0jMoKsA8cob9+NvfcQAb+fn/y4O+MBwCJCtdPBzIrSq9szGj37VxWTaR0AppryN9l87IkEnHjc4Asn5van8mkfYCAQ4dHXjvHw7nb6ApFTPzh+kMqu17APHU81jczVz5QZ5XZqXI6MFICbLophn4FSw/KdBppVuU75G2+nbiAco2swmCrGNpXB//hAkK3PvsP6bS/x4xeP0BeIYLMINyxZwE++1IS760UiXadPcY3M1Z+qcruVuuoK5s4o08H/DFrKQRWTab0GkA8jUxrPaTiX/+dr3+CD166c0hTPsHe6hmhu9fDswS6Ga7q5HFY+c+VCbr2qjjmViYye01IxKyoIBAJEIhHuueeeVLpmOkqxPn86NK1VFZqSXATOh0ylbw4zxvDq0T52tHpoe6831T6n0sFtV7m58YraURdex8vVnyyLCFVOO7Mq7DrPr1QRKslF4FwKRmIMBid/utZYYnHD79/qZkerh3e6hlLt58x2sr6pnlUXz8NuHXsGb8WKFVP6tD+sstxGjdOBbZznUkoVJw0AUxCPG4bCiSyeTHzaBwhEYjyx7zgPtHk5PhBMtV/hnsX6pnqubqjBkoNP4WV2K7Ndjowc7aiUKkwaANIQjMQYCEbwh2IZ+bQP0OcP88ieYzzyajsDwUQZZwGuvWAO6xrruXThzIw8z9nYLBaqXXZmlE/9MHelVGHTADBB4WgiZ98XmlrO/pmO9QV4oM3Lzv3HU3V+7FbhU5ctYM1yN/U1zrM8QmaICLMq7FRV2LFoZo9SJUEDwDhiccNQMMpgKJJ2EbaxvNU5yI5XPLS83Z3K6Kkss3HT0oXcsqyOGpcjo883ngqHldmustPq/iulpj8NAGcwxuALx/CFovjTOGDlbI/d9l4vO1o9vHq0L9U+b0YZty93c8PlC3A6cveWaPkGpUqb/uYnBcIxBkOZndcfFo3FeS6Z0XOo+1SRnnPnuFjbVM+qi+bmNMtG0zqVUlDiASASizMUTOTsZ3Jef1ggHOM3+zp4oM1L12Ao1b60fhbrmxbRtLg65wOwpnUqpYaVXACIxQ1DGdyoNZoeX5iHX23nsdeOMZjM6LEIrLhgLuua6rlowYysPO94NK1TKXWmkgkAw6mbU6mzfzbeXj8PtHl5Yv9xIrHEczhsFlZftoDbG93UVVVk5XnHo2mdSqmxlEwA6B4MZWWaB+BAxwDNrR6ef/sEw6FlZvmpjJ4qZ+4yeoZpWqdS6mxKJgBkmjGGlw/30Nzq4TVvf6p9/swy1iyvZ/XlC6jI03RLud3KnEpN61RKjU8DwCRFYnGefbOL5jYvh0+cyug5f24l65rcrLxoXt5KJOt0j1JqMvIaAETkeuB/AVbgXmPMd/LZn/H4w1F+vbeDB3e10z10KqNn+aIq1jXVs/yc3Gf0DNO0TqVUOvIWAETECvwA+ATgBVpF5DFjzBv56tNoTg6F+GUyo8cXSmQNWQRWXjSPdY1uLpif+4yekWZW2Kl26qlcSqnJy+cVwNXAO8aYQwAisgO4CSiIAHC0x8/9bR6eeqMzldFTZrNww+W13L68jtpZuc/oGcnpsFHjcug8v1IqbfkMAHWAZ8T3XuADZ95IRO4G7gZYtGhR1ju1/1g/O1o9/OGdk6mMnlkVdm5eupCbl9Yxy5nf+XW71cLsSkdOS0Yopaangh9FjDHbgG2QOBEsG88RN4aXDp2kudXD6+0DqfbaWeWsbaznU5fNz/sGKosI1U4HMytsOs+vlMqIfAaAdqB+xPfuZFvOhKNxnn6zi/tbPbzX40+1Xzi/kvVNi/jIBXMKYm59RrmdGpfO8yulMiufAaAVuEBEGkgM/OuBz+XiiYdCUR7f28FDu72cHAqn2q9eXM3apnqW1VcVxKdsp8NGtctOmU3LNyilMi9vAcAYExWRjcBvSaSB3meM2Z/N5+weDPHL3V5+tbcDf/hURs+qi+exrqme8+ZWZvPpJ6zMbqXG6aDCoQO/Uip78roGYIz5DfCbbD/P252DfP93b/Hk/k6iydNXyu0WPn15LbcvdzN/Znm2uzAhNouFmkoHlVqfXymVA9N+pDlywscnvt+S+r7aaeeWZXV89sqFzKwojB2zIkJVhZ0qp27kUkrlzrQPAIvnuLjmvNkc7fGzZrmbT122oKBy511liXx+u9bnV0rl2LQPAAD/+45lDAWjGT/payo0n18plW8lMfrMqSwjEI4Rj+U/AOh0j1KqUJREACgUOt2jlCokGgBywGGzMNtVpmmdSqmCogEgi6wWocrpYFaBZBsppdRIGgCyQESYWW6j2unQ4xiVUgVLA0CGaZlmpVSx0ACQIZrWqZQqNjpaTZGWaVZKFSsNAFOgZZqVUsVMA0Aayu1WZlc6tEyzUqqoaQCYBLvVQrVLq3UqpaYHHckmQMs3KKWmIw0AZ1FZbqPG6cCm5RuUUtOMBoAxlNmtzHY58n4YvFJKZYsGgDPYLBaqXXZmlGv5BqXU9JaXeQ0R+ZaItIvInuSfG/LRj5GG8/nrayp08FdKlYR8XgF83xjz3Tw+f4rO8yulSlFJTwFpPr9SqpTl8yPvRhHZKyL3iUj1WDcSkbtFpE1E2rq7uzPyxHarhXkzy1lYVaGDv1KqZInJ0jm5IvI7YMEoP/pr4CXgBGCAbwO1xpgvn+0xGxsbTVtbW1r98fT4icaN5vMrpUqOiOwyxjSe2Z61KSBjzMcncjsR+RHweLb6MczpsDKzwq7HMSqlVFK+soBqR3x7C7Av2885u7JMB3+llBohX4vA/yAiS0lMAR0BvpKnfiilVMnKSwAwxnwhH8+rlFLqFJ0TUUqpEqUBQCmlSpQGAKWUKlEaAJRSqkRpAFBKqRKlAUAppUqUBgCllCpRWasFlA0i0g34SNQRKnZz0NdRSPR1FBZ9HZl1jjFm7pmNRRUAAESkbbSiRsVGX0dh0ddRWPR15IZOASmlVInSAKCUUiWqGAPAtnx3IEP0dRQWfR2FRV9HDhTdGoBSSqnMKMYrAKWUUhmgAUAppUpUUQYAEfmWiLSLyJ7knxvy3afJEJHrReSgiLwjIn+Z7/6kS0SOiMjryfcgvcOa80BE7hORLhHZN6KtRkSeEpG3k39X57OPEzHG6yi63w0RqReRZ0XkDRHZLyJfS7YXzXsyzmso6PejKNcARORbwJAx5rv57stkiYgVeAv4BOAFWoE7jDFv5LVjaRCRI0CjMaYQNrpMmIisAIaAnxljliTb/gHoMcZ8JxmUq40xf5HPfp7NGK/jWxTZ70byiNhaY8xuEZkB7AJuBr5Ekbwn47yGtRTw+1GUVwBF7mrgHWPMIWNMGNgB3JTnPpUUY0wL0HNG803AT5Nf/5TEL29BG+N1FB1jTIcxZnfy60HgAFBHEb0n47yGglbMAWCjiOxNXgYX7KXhKOoAz4jvvRTBf5QxGOBJEdklInfnuzNTNN8Y05H8+jgwP5+dmaJi/d1ARBYDy4CXKdL35IzXAAX8fhRsABCR34nIvlH+3AT8EDgPWAp0AP+Yz76WsGuNMVcBq4GvJqckip5JzIsW39xoQtH+bohIJfAQ8HVjzMDInxXLezLKayjo9yMvh8JPhDHm4xO5nYj8CHg8y93JpHagfsT37mRb0THGtCf/7hKRh0lMb7Xkt1dp6xSRWmNMR3I+tyvfHUqHMaZz+Oti+t0QETuJgfM/jDG/TDYX1Xsy2mso9PejYK8AxpP8zzDsFmDfWLctQK3ABSLSICIOYD3wWJ77NGki4koudiEiLuCTFNf7cKbHgDuTX98JPJrHvqStGH83RESA7cABY8z3RvyoaN6TsV5Dob8fxZoF9G8kLqkMcAT4yoi5woKXTAX7J8AK3GeM+R/57dHkici5wMPJb23Az4vldYjIL4CVJEr1dgJ/CzwC3A8sAt4D1hpjCnqBdYzXsZIi+90QkWuB54HXgXiy+Zsk5tCL4j0Z5zXcQQG/H0UZAJRSSk1dUU4BKaWUmjoNAEopVaI0ACilVInSAKCUUiVKA4BSSpUoDQCq6IlILFlpcZ+IPCAizik81k9E5Pbk1/eKyKXj3HaliFyTxnMcEZE5Y/xsqYgYEbl+so+r1GRpAFDTQcAYszRZETMM/PHIH4pIWjvejTF3naVK60pg0gHgLO4AXkj+rVRWaQBQ083zwPnJT+fPi8hjwBsiYhWRLSLSmizM9RVI7OAUka2SOJ/hd8C84QcSkedEpDH59fUisltEXhORp5MFv/4Y+LPk1cdHRGSuiDyUfI5WEflw8r6zReTJZJ34ewEZrePJ3aRrSJRB/oSIlI/42d8k+/iCiPxCRP7fZPt5IvJEsiDf8yJyccb/RdW0VbC1gJSarOQn/dXAE8mmq4AlxpjDyWql/caYJhEpA14UkSdJVG28CLiURLXJN4D7znjcucCPgBXJx6oxxvSIyL8wota7iPwc+L4x5gURWQT8FriExA7dF4wxm0Xk08CGMV7CNcBhY8y7IvIc8GngIRFpAm4DrgTswG4S9eYhcej4Hxtj3haRDwD/DKxK859QlRgNAGo6qBCRPcmvnydRk+Ua4BVjzOFk+yeBK4bn94FZwAXACuAXxpgYcExEnhnl8T8ItAw/1jjlCD4OXJr4IA/AzGR1yBXArcn7/lpEese4/x0kzocg+fcXSRQX+zDwqDEmCARF5FeQqjx5DfDAiOcsG+OxlXofDQBqOggYY5aObEgOiL6RTcCfGGN+e8btMnlEnwX4YHKgPrMv45LESXG3ATeJyF+T6O/s4YJ74zxf35mvXamJ0jUAVSp+C/y3ZMleROTCZBXTFmBdco2gFvjYKPd9CVghIg3J+9Yk2weBkQP0k8CfDH8jIkuTX7YAn0u2rQZGOxTkOmCvMabeGLPYGHMOiU//twAvAp8RkfLkp/4bAZL15g+LyJrkY4uIXDmZfxRV2jQAqFJxL4n5/d2SOET9X0lcAT8MvJ382c+A/zzzjsaYbuBu4Jci8hrQnPzRr4BbhheBgT8FGpOLzG9wKhvp70gEkP0kpoKOjtK/OzhVXXXYQyTOi24lURp5L7CTRMXJ/uRtPg9sSPZrP3q8qJoErQaqVBEQkUpjzFByj0MLcPfwGbRKpUvXAJQqDtuSm9LKgZ/q4K8yQa8AlFKqROkagFJKlSgNAEopVaI0ACilVInSAKCUUiVKA4BSSpWo/wvsl2NcSi2sGAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# plot results\n", "sns.regplot(x=y_pred,y=y_test, scatter_kws=dict(color='k'))\n", "plt.xlabel('Predicted Age')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### [Cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics))\n", "\n", "Not perfect, but its not bad, as far as predicting with unseen data goes. Especially with a training sample of \"only\" 69 subjects.\n", "\n", "- But, can we do better?\n", "- On thing we could do is increase the size our training set while simultaneously reducing bias by instead using 10-fold **cross-validation**" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "
\"logo\"
\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Cross-validation is a technique used to protect against biases in a predictive model\n", "- particularly useful in cases where the amount of data may be limited. \n", "- basic idea: you partition the data in a fixed number of folds, run the analysis on each fold, and then average out the overall error estimate" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Let's look at the models performance across 10 folds" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "# import modules needed for cross-validation\n", "from sklearn.model_selection import cross_val_predict, cross_val_score\n", "\n", "# predict\n", "y_pred = cross_val_predict(lin_svr, X_train, y_train, cv=10)\n", "# scores\n", "acc = cross_val_score(lin_svr, X_train, y_train, cv=10)\n", "mae = cross_val_score(lin_svr, X_train, y_train, cv=10, \n", " scoring='neg_mean_absolute_error')\n", "# negative MAE is simply the negative of the \n", "# MAE (by definition a positive quantity), \n", "# since MAE is an error metric, i.e. the lower the better, \n", "# negative MAE is the opposite" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "scrolled": true, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fold 0 -- Acc = 0.419, MAE = 5.271\n", "Fold 1 -- Acc = 0.11, MAE = 3.069\n", "Fold 2 -- Acc = 0.79, MAE = 2.275\n", "Fold 3 -- Acc = 0.809, MAE = 3.061\n", "Fold 4 -- Acc = 0.641, MAE = 3.906\n", "Fold 5 -- Acc = 0.195, MAE = 4.732\n", "Fold 6 -- Acc = 0.684, MAE = 3.974\n", "Fold 7 -- Acc = 0.815, MAE = 2.693\n", "Fold 8 -- Acc = 0.058, MAE = 5.525\n", "Fold 9 -- Acc = 0.698, MAE = 2.571\n" ] } ], "source": [ "# print the results for each fold\n", "for i in range(10):\n", " print(\n", " 'Fold {} -- Acc = {}, MAE = {}'.format(i, np.round(acc[i], 3), np.round(-mae[i], 3))\n", " )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "For the visually oriented among us" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "Text(0, 0.5, 'Metric score [i.e., R-squared 0 to 1]')" ] }, "execution_count": 84, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAegAAAFzCAYAAAD18ZqMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABWCklEQVR4nO3dd3xUVfrH8c9JpSSEFloCJCT0FoqgIigSKVKVotJEXd3VddHdn72sus266rK6rroiHUSxAAIBCyoqSgsdEjpJKKEF0kg7vz/OBANCMgkzc+/MPO/Xa14h0+6XiHnmnnvOc5TWGiGEEELYS4DVAYQQQgjxa1KghRBCCBuSAi2EEELYkBRoIYQQwoakQAshhBA2JAVaCCGEsKEgqwOUVb9+fR0TE2N1DCGEEMIj1q1bd0xrHXmxx2xVoGNiYli7dq3VMYQQQgiPUErtv9RjMsQthBBC2JAUaCGEEMKGpEALIYQQNiQFWgghhLAhW00SE8JbFBYWkpaWRn5+vtVRRBmBgYHUrl2b+vXrExAg5x/Cu0mBFqIK0tLSCA8PJyYmBqWU1XEEoLWmsLCQI0eOkJaWRrNmzayOJMRlkY+YQlRBfn4+9erVk+JsI0opQkJCiIqKIicnx+o4Qlw2KdBCVJEUZ3uSoW3hK+RfshBCCGFDUqCFEF5l5cqVREdHWx1DCLeTAi2Ej4mJiaF69eqEhYXRqFEjJk2aRHZ2ttWxhBCVJAVaCB+0aNEisrOzSU5OZsOGDTz//PNWR7qk4uJiqyMIKxzZBiUlVqewNSnQQviwRo0aMWDAAJKTky/6+LRp02jRogXh4eHExsYye/ZswBTNhx56iPr169OiRQvefPNNlFIUFRUB5iz9iy++OPc+zz77LOPHjz/3/ejRo2nUqBERERH06dOHrVu3nnts0qRJ3Hvvvdx4443UrFmTr7/+moyMDEaOHElkZCSxsbFMmTLl3PPz8vKYNGkSderUoV27dqxZs8aVPyJhhUOb4K2rYPN8q5PYmqyDFuIyPbdoK9syTrv1GO2a1OKZoe0r/bq0tDSWLl3K9ddf/6vHcnJymDx5MmvWrKF169YcOnSIEydOAPDuu++yePFiNmzYQM2aNRk5cmSljjto0CCmTp1KSEgIjz76KOPGjTvvQ8KcOXNYsmQJixcvJj8/n969ezN8+HDmzp1LWloaiYmJtG7dmgEDBvDcc8+xe/dudu/eTU5ODoMGDar0z0HYzM4ljq9LofOt1maxMTmDFsIHjRgxgvDwcJo2bUqDBg147rnnLvq8gIAAtmzZQl5eHo0bN6Z9e/MhYP78+Tz44IM0bdqUunXr8vjjj1fq+HfeeSfh4eGEhoby7LPPsnHjRrKyss49Pnz4cHr16kVAQACbN28mMzOTP//5z4SEhNCiRQvuvvtu5s2bdy7Lk08+Sd26dWnatCmTJ0+u4k9F2EZKkvm652soLrI2i43JGbQQl6kqZ7bu9umnn5KYmMg333zD2LFjOXbsGI899hizZs0C4IknnuCJJ57ggw8+4JVXXuGuu+6iV69e/POf/6RNmzZkZGTQtGnTc+/XvHlzp49dXFzMk08+yYcffkhmZua5dcnHjh0jIiIC4Lz33r9/PxkZGdSuXfu89+jduzfAZWURNpR9FDLWQ6OOcHgzpK+DZj2tTmVLcgYthA+79tprmTRpEg899BD//e9/yc7OJjs7myeeeAKAAQMGsGLFCg4dOkSbNm24++67AWjcuDEHDx489z4HDhw4731r1qxJbm7uue8PHz587s9z5szhs88+44svviArK4t9+/YBphVnqbJNXpo2bUpsbCynTp06dztz5gxLlixxKovwMqkrzNf+fwcVALu+KP/5fkwKtBA+7sEHH2TFihVs3LjxvPuPHDnCZ599Rk5ODqGhoYSFhZ072x0zZgxTpkwhLS2NkydP8sILL5z32oSEBObNm0dhYSFr167lo48+OvfYmTNnCA0NpV69euTm5p77MHApPXr0IDw8nBdffJG8vDyKi4vZsmXLuclgY8aM4fnnn+fkyZOkpaXx73//2xU/FmGV1CQIbwyxfSD6CinQ5ZACLYSPi4yMZOLEifzlL3857/6SkhJeffVVmjRpQt26dfnmm2946623ALj77rsZMGAAnTt3pmvXrtx8883nvfavf/0ru3fvpk6dOjzzzDOMHTv23GMTJ06kefPmREVF0a5dO6688spy8wUGBrJ48WKSk5OJjY2lfv36/OY3vzl3zfqZZ56hefPmxMbG0r9/fyZMmOCKH4uwQlEB7P4aWvYHpSA+ETI2QM4xq5PZkio77GS17t2767Vr11odQ4gKbd++nbZt21odw2P27dtHbGwshYWFBAXZf+qKv/338Rp7voEZw+DWOdBmsLn+/O71cPP/oNNoq9NZQim1Tmvd/WKPyRm0EEIIz0hdDoEhEHut+b5xF6hRT4a5L0EKtBBCCM9ISYKYayA0zHwfEABx18PuL6Wr2EVIgRZCVCgmJgattVcMbwubOrEHjqdCywHn3x+fCDmZcHiTNblsTAq0EEII90tZbr626n/+/XGOLncyzP0rUqCFEEK4X2oS1G8FdVucf39YA2jcGXZ9aU0uG/PNAl1cCKteh22fWZ1ECCHE2WzYt8osr7qY+EQ4+BPknfJoLLvzzQIdEAQb58H3Uyp+rhBCCPfasxKKC6DVgIs/Hp8Iuhj2fuPRWHbnmwVaKegyDtLXQuZOq9MIIYR/S02C0FrQ7KqLPx59hXlcrkOfxzcLNEDHMaACIXm21UmEEMJ/aW36b8f1hcDgiz8nMBhaXGuuQ9uoeZbVfLdAhzc01zs2zpPtzIRfiYmJISQkhGPHzm+f2KVLF5RS5zavAHj22WdRSvHTTz+d99xp06YRGBhIWFjYebeMjAxP/BWELzm8Cc4c+vXyqgvFJ8LpdMjc4ZlcXsB3CzSYYe7sI7D7K6uTCOFRsbGxzJ0799z3mzdvPm/3KTC7S82YMYO6desyY8aMX73HVVdddW73q9JbkyZN3J5d+JjS5VUtbyj/efGJ5qsMc5/j2wW65QDTRi55ltVJhPCoCRMmnFd0p0+fzsSJE897znfffcehQ4eYMmUK8+bNo6CgwNMxhT9ITYKobmY5VXkioiGyrRToMny7LVBQiLkWvfY9yD0BNepanUj4oqWPmY3n3alRRxj0QsXPc7jyyiuZOXMm27dvp1WrVsybN4/vv/+ep5566txzpk+fztChQxkzZgyTJ09m0aJFjBw50h3phb/KOQZpa+G6x517fnw/+PkdKMiBkJruzeYFfPsMGswwd3EBbP6o4ucK4UNKz6JXrFhB27ZtiYqKOvdYbm4uH374IWPHjiU4OJhRo0b9aph79erV1K5d+9wtLi7O038F4e1SVwD6193DLiU+0fy+3rfKrbG8hW+fQYM582jU0Qxz97zH6jTCF1XizNaTJkyYQJ8+fdi7d++vhrc/+eQTgoKCuPHGGwEYN24ciYmJZGZmEhkZCZiz8FWr5BeluAypSRDWEBp1du75za6C4BpmmPtSa6b9iO+fQQMkjIdDG+HwFquTCOExzZs3JzY2liVLlnDzzTef99j06dPJzs6mWbNmNGrUiNGjR1NYWMicOXMsSit8TnEh7PrKTA4LcLLUBFeDmN5yHdrBPwp0x9EQEAzJ8stH+Jf33nuPr776ipo1f7mel56ezpdffsnixYtJTk4mOTmZjRs38uijj150NrcQVXLwJzibVfHyqgvFJzp2vtrtnlxexK0FWim1Tym1WSmVrJRa685jlatmPWg9EDZ9YD7VCeEn4uLi6N69+3n3fffddyQkJNC/f38aNWp07jZ58mQ2bdrEli1mpOnHH3/81TroNWvWWPHXEN4oJcmcGMX1rdzr4vuZr7I81iPXoPtqrY9V/DQ3SxgP2xdB6nJoM9jqNEK4TdlGJGUFBQWhHV2aHnvssV893qRJEwoLzQfYDh06MGnSJHdFFP4gJQliekFoeOVeVy8O6sSaYe4ed7snm5fwjyFuMMMmNRvABmn9KYQQbnVyHxzbWfnh7VLxibD3Wyg669JY3sbdBVoDy5VS65RS1k6hDgyCzreYWYXZmZZGEUIIn1baPayqM7HjE6EwFw786LpMXsjdBfoarXVXYBDwe6VUnwufoJS6Rym1Vim1NjPTzYUzYRyUFMHm+e49jhBC+LPUJKgbZ4arqyLmGggMcayj9l9uLdBa63TH16PAJ0CPizznHa11d61199L1l27ToC006WqGuWXHFCGEcL2CHNj73eWtYw4NM2uid33pulxeyG0FWilVUykVXvpnoD9g/ULkLuPg6FazLlqIy6DlQ54tlZSUWB3Bv+39ForPXn6jkfhEyNwOWWmuyeWF3HkG3RBYpZTaCPwMfK61XubG4zmnw0gIDJV9osVlqVatGsePH5cibSNaawoKCkhPTz9v3bfwsJRlEBIOza6+vPc5t7uV/55Fu22ZldZ6D+BkfzcPql7HLLPa/CH0/xsEhVqdSHih6Oho0tLScPu8CVEpQUFBREREUL9+fauj+CetzXXjuOvMZkWXo0FbCG9illt1u90l8byN7/fivpgu42Drx7BzKbQfYXUa4YWCg4OJjY21OoYQ9nJkC5xOd373qvIoZZqWbPvMNJgKDL789/Qy/rMOuqwWfc0nMxnmFkII10lJMl9bOrl7VUVa3gBnT5stK/2QfxbogEDofKsZOjlz2Oo0QgjhG1KXQ+MECG/omveLvRZUoN9unuGfBRrMmmhdAhvnWZ1ECCG8X85xSFvj2m0iq9eGpj2kQPud+vHQtKcZ5paZuEIIcXl2fWFOely9j3N8PziU7JcdIP23QIM5iz6WAunrrE4ihBDeLTXJ7HfQuItr37d0uZUf7m7l3wW6/U0QVB02zLI6iRBCeK/iInMG3fIGCHBxWWnUGWrU98thbv8u0NVqQbthsOVjKMyzOo0QQnintJ8hP8t1s7fLCggww9y7vwQ/6xLn3wUazDD32SzY8bnVSYTwLUVn4chWq1MIT0hJgoAgiOvrnvePT4Tc4+ZatB+RAh3TGyKayTC3EK5UkAOzRsJbV8O3L8tETF+XutxsblEtwj3vH3c9oPyu7acU6IAASLgN9qz066bsQrhMQQ7MHgP7v4fm18BXf4Olj/rd8KTfOHUAjm6DVgPdd4ya9aFJgt9dh5YCDdD5NkDDxrlWJxHCu53Nhtmj4cAPcPO7cPsiuOp++Plt+PhuKCqwOqFwtdLuYa5eXnWh+ERzrTvvpHuPYyNSoAHqxppP+slzZChOiKo6V5xXw8j/QcdRZoSq/98g8TnY8hHMvcU8T/iO1OVQJxbqxbv3OPGJZp31npXuPY6NSIEu1WUcnNhjfrkIISrn7BmYPQoO/mSKc4eRvzymFFzzIAx/E/Z8AzOGma5TwvsV5Jr9n1sNMP+d3SmqO4RG+NUwtxToUm2HQXBNSJbJYkJUSv5pMyHs4M8w6j3ocPPFn9dlPNwyy8zsnjrAXLsU3m3fd1CU757lVRcKDDLbWO760m9GOqVAlwoNM41Ltn5qJrkIISpWWpzT18Ho983/Q+VpcyNM+ASyj8J7A+Dods/kFO6RkmRObGKu8czx4hPhzCEzKc0PSIEuq8s4KMiGbQutTiKE/eVnwaybIWM9jHof2g137nXNr4Y7l5rriVMHwoGf3JtTuIfW5vpzi+sgKNQzx4zrZ776yTC3FOiyml1lJjvIPtFClC8/C2beDBkbYPQ005GvMhq2h7uWQ416MGP4LzOBhfc4ug2yDrp/9nZZEVHQoL0UaL+klOkstu87OLnP6jRC2FPeKZh5ExzaCGNmQNuhVXufOs3hziSIbA1zb4NkWeboVUo/VHni+nNZ8f1g/49+sRpACvSFOt8KKPllIcTFnCvOm0xxbjP48t4vLBImLYbY3vDp7+D7KS6JKTwgdTk06gS1Gnv2uPGJUFJoTqR8nBToC9VuCi2uNWuipfOREL/IOwkzR8DhzXDLTDPhyxVCw2HsfGh/M6x4GpY/7TezdL1W7gmzpM6Tw9ulml1pJqb5wTC3FOiLSRgPWQdg/yqrkwhhD3knYcYIs0TqllnQepBr3z8o1KyfvuJu+GEKfHofFBe69hjCdXZ/ZSb5tbSgQAeFQmwfSF3h8x/kpEBfTJvBEFoLNshkMSHIPWEmch3d5ijObuq5HBAIN74M1z0BG+fAvHGmEYawn5QkM8Evqqs1x4/vB6f2m+ZSPkwK9MWE1DDNFrZ9ZtZ5CuGvzhXnHXDrHPcPaSoF1z0Kg1811zhnjjAZhH2UFMOuFWZyWECgNRniE81XHx/mlgJ9KQnjoSgPtn1qdRIhrJF7wrTlzNxpinPLGzx37CvugjHTzTKu92+ErHTPHVuUL22NueTh6dnbZdWNhbpxUqD9VnR3qNdShrmFf8o5DtOHQWYK3DYHWiZ6PkO74TB+gdkGduoAk0VYLyUJVKBjj2YLxSfC3u+gMN/aHG4kBfpSlDKdxQ6uhuO7rU4jhOfkHDdnzsdT4ba5vwwnWiG2D9zxuen3PHUApK2zLoswUpebpk7Va1ubIz7RjHIe+MHaHG4kBbo8nW4FFSCdxYT/yDkG04fC8V1w2zwzGcdqjTubhibVaplsu760OpH/ykqDI1uglYXD26ViekFgKKT67jB30KUeUEr9yYnX52it33ZhHnup1dj0fk2eC32ftG5ChBCekJ1pzpxP7IWxH5gey3ZRL84U6VmjYM4tcNN/zX7TwrNSl5uvViyvulBITdPXfdcXwD+sTuMW5Z1BPwyEAeHl3P7P3QEt12UcnMnwq03ChR/KzjRnp3YszqXCG5nh7qY9YMFdsPq/VifyPylJULu5ac9qB/GJcGynz25deskzaGCm1vov5b1YKVXTxXnsp9UgqFbbDHPbYbhPCFfLPmqK88n9MG6+ue5rV9UiYPzHpkAvexRyMuH6p8ycEeFehXmw5xvoOsE+P+/4RFj+pLns0f0Oq9O43CXPoLXWj1T0Ymee4/WCq0HH0bB9selDLIQvOXMEpg0xZyDjPrR3cS4VXA1GT4euE+G7V2DRA1BcZHUq37dvlZmUZYfh7VKRraFWtM8ut6rSJDGllO99VClPl3FQfBa2LLA6iRCuc+YwTB9iJv6M+8hsWOEtAoNg6BTo/RCsnw4f3u7Ty21sISUJgmtAzDVWJ/mFUmZkc883PtkatqqzuJ9zaQq7a5wADdrJbG7hO84cNmfOWekw/iMzI9bbKAX9noZBL8GOxTBrpNmnWrie1pCaBLHXmhEMO2l5AxScgYM/W53E5S5ZoJVSmy5x2ww09GBG65XuE52+zrQ8FMKbnT4E0wbDmUOmEUjzq61OdHl6/hZGvmd2V3p/sBm2F66V6ZiIZYflVReK7QMBQT45zF3eGXRDYCIw9CK34+6PZjOdxpjuOXIWLbzZ6QxHcT7sKM5XWZ3INTqOMrPPT+yBqf19fhMFj0tZZr5a2d7zUqpFQNOeflegFwNhWuv9F9z2ASs9ks5OwhqYjQI2fSATUoR3yko3xTn7qJkJ3exKqxO5Vnw/uH2R2eDmvf5waKPViXxH6nJo2BEioq1OcnHx/eDwJp8bPSlvFvddWuuLboistR7rvkg2ljAOso/AbulkJLxMVpqjOGfChI+hWU+rE7lHdDfT0CSomhnu3vut1Ym8X95JOLDansPbpUrb0e7+ytocLiatPiujZX+zB+qGWVYnEcJ5pcU59zhM+MQ0+vBlka1MkY6INhPHtn1mdSLvtvsr0MX2Wl51oYYdoWYDnxvmlgJdGUEh0OkW2LlU9qgV3uHUQUdxPuEozldYncgzIqLgjiXQpAvMvx3WTrU6kfdKWQ7V65od/uwqIMAMc+/+yuxX7SOkQFdWwjgoKYTNH1qdRIjynTrgKM4nYcKn9v4F6w416pq/d8v+sPiPsPJFs1xIOK+kGHatMEPIdt+LID4R8k5ARrLVSVzGqQKtlGqolBriuDVwdyhba9QBGnWSYW5hbyf3m+KcdwomfmKuzfqjkBpw62zoPBZW/gOWPAwlJVan8h7p68ylkVY2Ht4u1aIvoHxqmLvCAq2UGgP8DIwGxgA/KaX8exuZLuPNjMHDm61OIsSvndxvmpDkZ8HETyHKT4tzqcBgGPEfuPoPsOZd08e76KzVqbxDSpJZXhp3vdVJKlazHkR19a8CDTwJXKG1vl1rPRHoATzt3lg212EUBARD8hyrkwhxvpP7zJnz2dMwcaH5hSVMs6H+f4Mb/gJbP4Y5Y+DsGatT2V9qklljXKOu1UmcE58I6Wt9Zo6QMwU6QGt9tMz3x518ne+qWQ9aDzJroosKrE4jhHFirzlzPnsGJn4GTRKsTmQ/vR6AEW/B3u/MDl45x6xOZF+nM8wooZ2XV10oPhF0Cez52uokLuFMoV2mlEpSSk1SSk0CPgeWujeWF+gy3lybKd3AXAgrndhjinNBNty+UIpzeRLGwq1zTNve9/qbSwLi10p/t9l5edWFmnQ12wPv8o1eFRUWaK31w8DbQCfH7Z3KbDOplApUSm1QSi2uekwbiusHYQ2l9aew3vHdpjgX5ppOWo07W53I/loPNKMMucdMkT6y1epE9pOyHCKaQoO2VidxXmAQxPU116F9YMa+M5PEXtRaf6y1/pPj9olS6sVKHOMBYHvVI9pUYJBZE52SZFonCmGFc8U5z5w5N+podSLv0awn3LHMXJ9+fxDs/9HqRPZRmA97VpolakpZnaZy4hNNx8cjW6xOctmcGeK+4SL3DXLmzZVS0cBg4H+VCeU1EsaZDjub5ludRPij47vNhLDis+bMWYpz5TVsB3ctN12oZo4wTYgE7F8FhTnesbzqQnH9zFcfmM1d3naT9zq2lmx9wXaTe4FNTr7/68AjwCUXHiql7lFKrVVKrc3MzKxMdus1aGOWsCTP9onhFOFFju2C9280m9TfvtiszxdVU7uZaQ3aoB3MGyc9DsAMbwdVN1s5eptajU3rTx+4Dl3eGfQczNaSCzl/q8luWuvxFb2xUmoIcFRrva6852mt39Fad9dad4+MjHQ+uV0kjIOj2+BQstVJhL84lmrOnEuKzJlzw3ZWJ/J+NeuZn2VsH/js97DbN2YBV4nWZnlVbB8Irm51mqqJ7wcHfvT6pXTl7WaVpbXep7W+7YLtJp1dYNYLGKaU2gfMA65XSvneR9MON0NgKGyQyWLCAzJTTHHWxTBpsRRnVwoNg9vmQb14WDQZCnKsTmSNY6lmPb03La+6UHyi+QDr5buZuW09s9b6ca11tNY6BrgV+MqZM2+vU70OtB1ienNLdyLhTpk7HcVZm2Ftb5pd6y2Cq8GwN0wf86/+ZnUaa6Qmma/etLzqQk17QkiY11+H9u+GI66SMA7yT8HOJVYnEb4qc6eZrQ3mzLlBG2vz+LLmV8EVd8Pqt+DgGqvTeF6K43p87aZWJ6m6oBCIvdbrl1s5s8yqoVKqq+PWsCoH0Vqv1FoPqcprvUKL66BWlAxzC/fITDHFWSmY9DlEtrY6ke9LfMbsJ73wfv8aGcvPMtduW3rx8Hap+H5mJOT4LquTVFl5s7gTlFKrgZXAS47bN0qp1UopafBbVkAgdL4Vdn8Jpw9ZnUb4kuO7TUtKMMPaka2szeMvQsNhyOuQuQO++6fVaTxn91fm2m2rgVYnuXzx3r/cqrwz6GnAA1rrtlrrRMetDfAg8L4nwnmVhHGmB+ymeVYnEb6itH1nSZFpQiLF2bNaJkLn20yBPuz9TS+ckrLctMqMvsLqJJevTgzUa+mzBbqm1vqnC+/UWq8GarovkpeqFwdNrzTD3F58zUPYxMl9MG0oFOWblpQyIcwaA/5hJoIuvB+Ki6xO414lJbBrhZkBHRhkdRrXiE+EfatMpz0vVF6BXqqU+lwpdYtS6mrH7Ral1OfAMk8F9CpdxsHxVEhba3US4c1OHTDFuSDbFGdpQmKdGnXhxpchYwOs/o/VadwrYwPkZHpn97BLiU80H3L3f291kiopbx30ZOANoC/wuOPWF3hTa32/Z+J5mXYjTPedZN9b7i08JCvNsWVklinOjTtZnUi0GwFthsDXfzdzAnxVahKoAFPUfEVMLwiqBqneOcxd7ixurfVSrfXvtNZDHbffaa1lLdGlVKsF7YbDlo+9dkhFWOh0hinOeSdhwieyZaRdKAU3vmIaEi2cbIaCfVHKMnPtuUZdq5O4TnB1aN7La69DyzpoV+syDs6ehu2+tbumcLMzh01xzjkG4z82Pd6FfdRqDAP+ZjaRWD/d6jSud+YwHNroG8urLhSfaC49ntxndZJKkwLtas2vMc33ZZhbOCv7qFlKlX0Exi+Apj4wg9YXdZlg+lOv+DNkpVudxrVSl5uvvrC86kKlQ/ZeuHmGFGhXCwiAzmNhzzdw6qDVaYTdZWea4pyVBuM+NHsUC3tSCoZOMcvePv+Tb63WSEkyzZYatrc6ievVbwkRzbyyQF9yLr1S6t/AJf8FOiaRiYtJuA2+eQE2zoNrH7Y6jbCrnOMwYzic3G+Kc/OrrU4kKlI3Fq5/GpIehy0LoOMoqxNdvqKzsGcldBxtPoT4GqVM05LNH0JRgWkD6iXKO4NeC6wDqgFdgVTHLQHwnr+hFerEQExv2SdaXFruCVOcT+yGsfMgtrfViYSzev4WorrD0kfMnAFvt/8Hs6TPl5ZXXajlDebvePBXrT1srbxlVtO11tOBTsB1Wut/a63/DfTDFGlRnoRxcHKv6WsrRFl5J2HmCDiWArfOMb3chfcICIThb0D+aVj2mNVpLl9KkpmhHtvH6iTuE9sHAoK8bja3M9eg6wC1ynwf5rhPlKfdMLPdmWygIcrKOwUzb4Kj2+GWWb/0CxbepUFb6POwGTbd6eV9m1KTzAhOiA83iAwNh2ZXed11aGcK9AvABqXUNKXUdGA98A/3xvIBITWh/QjY+gmczbY6jbCD/NMwa6Tp6zxmBrTywSUt/uSaP5ptGRf/0fy39UbHdpme7744e/tC8f3gyGav2tCowgKttX4f6Al8AnwMXOUY+hYVSRgPhTmwfaHVSYTVzp6B2aPgUDKMngatB1mdSFyuoBAY9gZkH4YvnrE6TdWkJpmvvrj++UKly612f2VtjkpwZj9oBSQCnbXWnwEhSqkebk/mC5pdCXVbyDC3vyvIgdljTI/2UVOhre9uje53orvBlffB2qlmUwZvk5IEkW2gTnOrk7hfww4Q1tCrrkM7M8T9H+Aq4DbH92eAN92WyJcoBQljTfehE3utTiOsUJALc26Bg6th5P9MK1jhW/o+aVZuLPyDd7X4zT9tZnD7w9kzOJZbJTr2vC62Oo1TnCnQPbXWvwfyAbTWJ5FlVs7rfBugYONcq5MITyvMg7m3mp10bnoHOtxsdSLhDiE1YNi/zbXclc9bncZ5e76GkkLfXl51ofh+kH8K0tdbncQpzhToQqVUII6mJUqpSMBHu8W7QUS0WUaTPNd3m+yLXyvMh3ljYe+3MPw/0Gm01YmEO8X2gW6T4Id/e80vf1KWQ2gENPWj7nUt+podu7xkmNuZAj0FM0GsgVLq78AqZBZ35XQZD1kHYN93VicRnlB0Fj4Yb4bShr9hOssJ33fDX8w1zs/uNx2r7KykxPTfju8HgcFWp/GcGnXNRjS+UKCVUgHAXuAR4HngEDBCa/2hB7L5jjaDzSfVZJks5vOKCmD+RNi1Aob+y3w4E/6hWgQMeQ2OboXv/2V1mvIdSoaco/41vF0qPhHS15lufjZX0X7QJcCbWusdWus3tdZvaK23eyib7wiubq4/blvoveslRcWKC+HDSWZf3cGvmiFP4V9aD4IOI+Hbl+DoDqvTXFrqckD9svTIn8QnAtorlls5M8T9pVJqpGO5laiqLuOhKM80LhG+p7gQProTdn4Og16GK+6yOpGwysAXTRfBhX+w72zhlCSI7g4161udxPOadIHqdbximNuZAv1b4EPgrFLqtFLqjFJKTgMrK6ob1G8lw9y+qLgIPr7HNKQZ8Dz0vMfqRMJKYZEw6EVI+xl+ftfqNL+WfRQy1kNLPxzeBtNLPe560/bT5hN3nekkFq61DtBah2itazm+r1XR68QFlDIbaBz8ybTXE76hpBg+/R1s/Rj6/w2uus/qRMIOOo4264u/fM5sJ2onqcvNV39uNRufaK7BH9lsdZJyOXMGjVKqjlKqh1KqT+nN3cF8UqdbzBR/OYv2DSXF8Ol9ZsOEfs/A1X+wOpGwC6XMhDEVAIsesNe2sylJEN4YGnWyOol14hyb1Nh8mNuZVp+/Ab4FkoDnHF+fdW8sH1WrsfnktnGefa9NCeeUlMDCybBpHvR9Cnr/yepEwm4iouGG50xDELt8KC8qgN1fm7N7f55WFN7QfECx+e5WzpxBPwBcAezXWvcFugCn3BnKpyWMgzMZ5n9a4Z1KSmDxA5A8C659DK592OpEwq663QnNe0HSE3DmsNVpzP70BWf8c3nVheITzSXH/Cyrk1ySMwU6X2udD6CUCtVa7wBauzeWD2s9yMwglA00vJPWsOT/YP0M6P0QXPeY1YmEnQUEwNAppnnNkoesTmOuPweGQOy1ViexXnwilBSZbn825UyBTlNK1QY+BVYopT4DbDbrwYsEhZoJJDs+h7yTVqcRlaE1LH3E7FzU60G4/in/HiYUzqkfD9c9DtsXwbbPrM2SkgQx10BomLU57KBpDwgJt/V1aGdmcd+ktT6ltX4WeBp4Dxjh5ly+LWEcFJ+FLQusTiKcpbUZpvz5Hbjqfkh8VoqzcN5V90PjzvD5Q9Z1sDq+G46n+u/yqgsFBkOLa811aDtN4ivDmUlizUpvmLafyUAjdwfzaY07Q4P2MsztLbSGFU/D6v9Az3vNciopzqIyAoNg+JuQdwKWP2VNBlle9WvxiZB1EI6lWJ3kopwZ4v4cWOz4+iWwB1jqzlA+TynoMs40CzgqnVNtTWuzlvWHf8MVd8PA56U4i6pp1NFcGkmebc3s4ZQk0yypbgvPH9uu4u293MqZIe6OWutOjq8tgR7Aj+6P5uM6joGAIPssvxAX9/U/YNVr0O0OuPFlKc7i8vR52BTJRQ/C2WzPHfdsttmXvKWcPZ+ndjOo39p7C/SFtNbrAT/aQNRNwiLNtaCNH5g+zsJ+Vr5oNj3oMsFsfiHFWVyu4Gow7A0zrPrlXzx33D0robhAllddTHwi7PseCnKtTvIrzlyD/lOZ20NKqTlAhgey+b4u40y7OZsvlvdL374MK/9hJvQNnWKWywjhCs16Qs/fmgmHB1Z75pipSRBaC5pd5ZnjeZP4fmbS7v7vrU7yK8781gkvcwvFXIse7s5QfqNlf6hR3zS8EPax6jX46m+mNeuwf0txFq53/dMQ0dTseFWY795jaQ0pyyGur5m5LM7XvBcEVbflMLcz16CfK3P7u9Z6dmnjEnGZAoNNEdi5DHKOW51GgJkM9sWz0GEUjHjL7HwjhKuFhsHQ183s4W9fdu+xDm2E7MOyvOpSgquZteGpK6xO8ivODHEvUkotvNTNEyF9WsJYKCk0Gy4Ia61+yyyBaTcCbnpbirNwr/h+5hLK96/DoU3uO07p8qqWN7jvGN4uPhFO7IYTe6xOch5nxu72AHnAu45bNrAb+KfjJi5How5mXbQMc1vr53dh2WPQdiiM/J9ZtyqEu/X/G1SvCwvvN/uKu0NKktmPPqyBe97fF8Qnmq82mw/kTIHupbW+RWu9yHEbC/TWWn+jtf7G3QH9QsJ4OLzZvZ+ixaWtnWr6JLceDCOnynU64Tk16sLgV8ww9I9vuP79c45B+joZ3q5IvTio3dwrC3RNpdS5le1KqVigpvsi+aGOo0wD++Q5VifxP+tnwOI/ml9go9+HoBCrEwl/0264GblZ+Twc2+Xa905dAWjpHlYRpcxZ9N5vzcYmNuFMgf4jsFIptVIp9Q3wNWYLSuEqNeqaXa42zzf7tQrP2DDb7OkcnwhjZpiNTISwwo2vmH9/C/9gtjN1lZRlENYQGnV23Xv6qpY3QGGO55a+OcGZWdzLgJaYojwZaK21Xu7uYH4nYTzkHjfrFYX7bfwAPvs9tLgObpllZnIKYZXwRjDgH3DgB1g31TXvWVwIu78yhUeWClYspjcEBNtquZUzs7hHAyFa643AUGCuUqqr25P5m7jrIawRrHzBut1u/MWmD+HT30Fsb7h1DgRXtzqREGZGd4u+sOIZOHXw8t/vwGo4e1quPzsrNAyaX2Wr69DOfKx6Wmt9Ril1DdAPs93kWxW9SClVTSn1s1Jqo1Jqq1LqucsN69MCg2D4G3AsFaYNhjNHrE7km9a+Dx/fbZoT3DYPQmpYnUgIQykY+i/TWGTxHy9/C8TUJHNGGNfXNfn8QXwiHN0Kp+3RLNOZAl3s+DoYeFdr/TngzEyas8D1WuvOQAIwUCl1ZZVS+ouWN8C4+XByH7w/CLLSrE7kW374Nyx+0PFz/hBCZK6jsJk6zaHfn2HXisvvjZCyHGJ6QWi4a7L5A5stt3KmQKcrpd4GbgGWKKVCnXmdNkq3awl23Oy5K7adtLgOJnwCOZkwdZDtFs57Ja3h6+d/aUJyy2wZ1hb21eNuiO4BSx+F7MyqvceJvXBspwxvV1aDdhDe2DbXoZ0p0GOAJGCA1voUUBd42Jk3V0oFKqWSgaPACq31Txd5zj1KqbVKqbWZmVX8x+hrml0Jty+EgjOmSB/dYXUi76U1JD0J37xgJuKNmipLqYS9BQSaHvAF2bDs0aq9R2n3MNm9qnKUMh3e9nztvsYxleDMmXCu1vpjrXWqUuoerfUhZ2dxa62LtdYJQDTQQynV4SLPeUdr3V1r3T0yMrLSfwGf1aQLTFoCugSm3ShNTKqipBgWTYbVb0LP3zk2vpD2ncILNGgDfR6BLQtgx5LKvz4lCerGmQYconLiEyE/yzR4sVhl597/rioHcZx5fw0MrMrr/VbDdnDnMrPTyvQhcHCN1Ym8R3EhLPiNaUTS52EY+IIsNRHepdcD0LADfP4nyDvl/OsKcmDfKjl7rqoW14EKsMUwd2V/Yzm9Y71SKlIpVdvx5+rADYCM1VZWvTi4c6np1ztzBOz9zupE9leYBx+Mh60fQ+JzcP1TZuhKCG8SFGJGfbKPwIo/O/+6Pd+Y/Y2lQFdN9ToQfYVXFuihAEqpXk48tzHwtVJqE7AGcw16cSWPJwBqN4M7lkKtKJg9ClKt/4djW2ezYfZoM8Q3+FW45kGrEwlRdVFd4ar7Yf10U3idkZoEIeHQ7Gr3ZvNl8YmQscH0MrfQJQu0Y4LXbUqph8pcO05QSv0AVNjVXWu9SWvdRWvdSWvdQWv9F1eF9ku1GsMdS6B+S5h7K2xfZHUi+8k7aUYZ9v8AN78DV9xldSIhLl/fJ6BuCzOfoiC3/Odqbfpvx10nkyEvR3w/QJtObBYq7wz6PeA3QD1gilJqFvAK8JLWuosnwokL1KwPty+GJgkw/3bYNN/qRPaRfRSmDTG7Ao2ZAZ3GWJ1ICNcIrm6Guk/ug6//Xv5zj2yB0+myvOpyNe4CNepZPsxd3qa33YFOWusSpVQ14DAQp7U+7plo4qKq1zbrpOfeBh/fA4W50G2S1amsdeqgOXM+nQFjPzBtU4XwJTHXQPc7YfV/oP3NEN3t4s9LWWa+tpTdqy5LQID5PbLrS7N5iUUTTMs7aoHWugRAa50P7JHibBOh4aYTVnwiLHoAfvyP1Ymsc3y36bqWfdR8cJHiLHxV4nOmicbC+y+9613KcmicAOENPRrNJ8UnQu4xOLzRsgjlFeg2SqlNjtvmMt9vdkz8ElYKrg63zjb7yCY9Dt++bHUizzuyzRTnwly4fZFp8CKEr6pWC4a8Bke3warXfv14znFIWyOzt12l9MO+hcPc5Q1xt/VYClE1QaEwahp8dh989TczgaTfn/1jSVH6Opg1EoKqmRnuka2tTiSE+7UaAB1Hmw/kbYeaXgmldn0BaCnQrhLWwIxG7PrS9FKwwCXPoLXW+8u7eTKkKEdgEIz4r7kOvepVWPaYazd8t6N9q2D6MAitZRq5SHEW/mTgC+ZseuH9plteqdQkqNnATHASrhGfCAd/rlyjGBeS1kq+ICAAhrwOV94HP/3XLMco+z+uL0ldYc6ca0WZ4lwnxupEQnhWzfow6CUzivTTf819xUXmDLrlDdIxz5XiE0EXw14n16C7mPyX9BVKwYB/mKGYDTPNDO/iQqtTudZWx+z1+q3MmvBaTaxOJIQ1OoyEVgPhy7+anasO/mT6R8vsbdeKvgJCIyy7Di0F2pcoZdpa9nsGtnxk1koXnbU6lWtsmA0f3QlR3WDSYnMWIYS/Usp0ygsMNiNmKcsgIAji+lqdzLcEBkGLa811aO353ZIrXaCVUtOVUm9dbGcqYRO9/2SGwHZ+brqOVdR9yO5+ettMhGtxHUz4GKpFWJ1ICOtFRMENf4G938LP70Czq+T/DXeITzTNXzI9v5VEVc6g3wC+ACa4OItwpZ6/hWFvwJ6V5ppt/mmrE1We1vDtK7D0EWgzBG6bByE1rU4lhH10vR1iekNRvhnyFq4X3898tWCYu9IFWmu9Rmu9QGtdxZ3Ehcd0nQAj/wdpP5tOW7knrE7kPK3hi2fhq79Cp1tg9HSzrEwI8YuAANMGtOUAc11auF5ENES29Y4CDaCUusfVQYSbdBgJY2bC4c0wfShkZ1qdqGIlJfD5/8H3r0P3u8wyssDyluwL4cfqxsK4+WZDHeEe8f3MJjwFOR49bFUniflBJwwf0uZG06O6tC3m6QyrE11acRF8ei+sfc9sWD/4n7JsRAhhrfhEKC4wPRg8qEq/+bTWb7s6iHCzuOvNBKszh2HqQLMzjt0UnYUPb4dN8+D6p03vYX/oiiaEsLdmV0FwDY8Pc1dYoJVSDZVS7ymlljq+b6eUko12vVHzq+H2z8x6yamD4Fiq1Yl+UZBjZpzvWGxmoPd5SIqzEMIegquZyXipKzx6WGfOoKcBSUBpV4gU4EE35RHuFtUNJn0OJYVmuPvwFqsTmQ8MM282M86Hv2lmoAshhJ3EJ8LJveZSoYc4U6Dra63nA6VbTxYBPtpH0k806mA2mAgIhmmDTctAq+QcM5PX0tfBqKnQZbx1WYQQ4lLOLbf60mOHdKZA5yil6gEaQCl1JZDl1lTC/eq3hDuXmsYG04ebGYqedvoQvH8jZO6E2+ZC+5s8n0EIIZxRL85cfvPgnvPOFOg/AQuBOKXU98AM4A9uTSU8o06M2XCiVmMzxLz7K88d++Q+eH+gmVE+foFp8i+EEHbW87dQP95jh6uwQGut1wPXAlcDvwXaa603uTuY8JBaTWDSEvPpcM4tsGOJ+4+ZudPMJM87ZSatxVzj/mMKIYSXcWqZlda6SGu9VWu9RWtdqJRq5O5gwoPCIuH2RdCoI3wwHjZ/5L5jHdpoJqeVFJsdqaK6ue9YQgjhxaraAeI9l6YQ1qtRFyZ8Cs2uhAW/gfUzXX+MA6th2lCznvDOZdCwveuPIYQQPqKqjUoGuzqIsIFqtWDcR2bLuoX3w0/vuO69d38FM28yZ+t3LjND6kIIIS7pkgVaKVXL8bXuxW6eiyg8KqSG2TWq9WBY+jCseu3y33PH5+b6dt0WZnlXRPTlv6cQQvi48nYgmAMMAdZhlliVbeukgRZuzCWsFBQKY6bDJ78zO0oV5EDfJ6vW2WvTfPM+TbrA+I+geh2XxxVCCF90yQKttR7i+BrruTjCNgKD4eZ3ILg6fPsyFOTCgL9XrkivnQqL/2Rmad82F0LD3ZdXCCF8zCULtFKqkdb6cHkvduY5wosFBMLQKRBSE1a/CYU5MPg153aX+v5fsOLPZp/aMdNNoRdCCOG08oa4lwBdK3i9M88R3iwgAAa+YGZer3oVCvNg+H8uvT+z1vD1P+Dbl6D9zeYsPDDYs5mFEMIHlFegOyulTpfzuALKe1z4CqUg8RlzJv3VX6EwF0ZOhaCQ859XUgJJT8BPb0HXiTDkdXMWLoQQotLKuwYtv1nF+fo8ZM6kkx6HeWPhlpm/DF2XFMOiybBhFlz5+8pfrxZCCHGeqjYqEf7qqvtg6L/MxuWzR8PZM1BUAAvuMsX52sekOAshhAuUN8QtxMV1m2TOpD/5nWk+Ur0OpC6H/n+Dq2UfFSGEcAUp0KJqOo0xw9sf3gElReZ6c/c7rE4lhBA+w6kCrZS6BmiptX5fKRUJhGmt97o3mrC9tkNNZ7CCbNMeVAghhMtUWKCVUs8A3YHWwPtAMDAL6OXeaMIrNL3C6gRCCOGTnJkkdhMwDMgB0FpnANISSgghhHAjZwp0gdZaY/pvo5Sq6d5IQgghhHCmQM9XSr0N1FZK3Q18Abzr3lhCCCGEfyv3GrRSSgEfAG0wXcNaA3/WWq/wQDYhhBDCb5VboLXWWim1RGvdEZCiLIQQQniIM0Pc65VSMlVXCCGE8CBn1kH3BMYppfZjZnIrzMl1J7cmE0IIIfyYMwV6gNtTCCGEEOI8FQ5xa633A7WBoY5bbcd9QgghhHCTCgu0UuoBYDbQwHGbpZSqcEcEpVRTpdTXSqltSqmtjvcRQgghhBOcGeK+C+iptc4BUEq9CPwI/LuC1xUB/6e1Xq+UCgfWKaVWaK23XVZiIYQQwg84M4tbAcVlvi923FcurfUhrfV6x5/PANuBqKqEFEIIIfyNM2fQ7wM/KaU+cXw/AnivMgdRSsUAXYCfKvM6IYQQwl9VWKC11q8qpVYC1zjuukNrvcHZAyilwoAFwINa69MXefwe4B6AZs2aOfu2QgghhE9zZrvJK4GtpcPVSqlaSqmeWusKz4aVUsGY4jxba/3xxZ6jtX4HeAege/fuujLhhRBCCF/lzDXot4DsMt9nO+4rl6OP93vAdq31q1WLJ4QQQvgnpyaJObabBEBrXYJz1657AROA65VSyY7bjVXMKYQQQvgVZwrtHqXUZH45a74P2FPRi7TWq3BitrcQQgghfs2ZM+jfAVcD6UAapjf3Pe4MJYQQQvg7Z2ZxHwVu9UAWIYQQQjg40+rzJcfM7WCl1JdKqUyl1HhPhBNCCCH8lTND3P0d65eHAPuAeOBhd4YSQggh/J0zBbp0GHww8KHWOsuNeYQQQgiBc7O4FyuldgB5wL1KqUgg372xhBBCCP/mzH7Qj2FmcXfXWhcCucBwdwcTQggh/JkzZ9BorU+U+XMOkOO2REIIIYRw6hq0EEIIITxMCrQQQghhQ86sg75JKRVR5vvaSqkRbk0lhBBC+DlnzqCfKbu0Smt9CnjGbYmEEEII4VSBvthznJpcJoQQQoiqcaZAr1VKvaqUinPcXgXWuTuYEEII4c+cKdB/AAqADxy3s8Dv3RlKCCGE8HfO7GaVAzzmgSxCCCGEcLhkgVZKva61flAptQjQFz6utR7m1mRCCCGEHyvvDHqm4+srnggihBBCiF9cskBrrdcppQKBe7TW4zyYSQghhPB75U4S01oXA82VUiEeyiOEEEIInFvPvAf4Xim1kDKbZGitX3VbKiGEEMLPOVOgdztuAUC4475fTRoTQgghhOs4U6C3aa0/LHuHUmq0m/IIIYQQAucalTzu5H1CCCGEcJHy1kEPAm4EopRSU8o8VAsocncwIYQQwp+VN8SdAawFhnF+7+0zwB/dGUoIIYTwd+Wtg94IbFRKzXE8r5nWeqfHkgkhhBB+zJlr0AOBZGAZgFIqwbHkSgghhBBu4kyBfhboAZwC0FonA7FuSySEEEIIpwp0odY664L7ZB20EEII4UbOrIPeqpQaCwQqpVoCk4Ef3BtLCCGE8G/OnEH/AWgPnAXmAqeBB92YSQghhPB7FZ5Ba61zgScdNyGEEEJ4QHmNSsqdqa21Hub6OEIIIYSA8s+grwIOYoa1fwKURxIJIYQQotwC3Qi4AbgNGAt8DszVWm/1RDAhhBDCn11ykpjWulhrvUxrfTtwJbALWKmUut9j6YQQQgg/Ve4kMaVUKDAYcxYdA0wBPnF/LCGEEMK/lTdJbAbQAVgCPKe13uKxVEIIIYSfK+8MejyQAzwATFbq3BwxBWitdS03ZxNCCCH8Vnm7WTnTxEQIIYQQbiBFWAghhLAhKdBCCCGEDUmBFkIIIWxICrQQQghhQ24r0EqpqUqpo0opWZ4lhBBCVJI7z6CnAQPd+P5CCCGEz3JbgdZafwuccNf7CyGEEL5MrkELIYQQNmR5gVZK3aOUWquUWpuZmWl1HCGEEMIWLC/QWut3tNbdtdbdIyMjrY4jhBBC2ILlBVoIIYQQv+bOZVZzgR+B1kqpNKXUXe46lhBCCOFryt0P+nJorW9z13sLIYQQvk6GuIUQQggbkgIthBBC2JAUaCGEEMKGpEALIYQQNuS2SWJCXI7iEk1hcYnjpikqLqHA8efC4hIKikooKn1OkXmsqPSxMq8x3//yvELHa2qGBNIhKoKOURHUCwu1+q8rhNNO5BSwOzOb3Uez2XU0m92Z2ZzKK+TRgW24skU9q+MJF5ICLS7L97uO8W1qJoVF5xfUX/58/vcFxZrCohKKSsz9BUXm/qISfa7QFhaXUKLdlzk4UFFY/MsBompXp1N0BB2jTcHuGBVB7Roh7gsgRAWKSzQZp/LOFeDdmaXFOIcTOQXnnlctOIAW9cPIyitk4tSfmXJrFwZ2aGRhcuFKUqBFlWit+c/K3byyfCdBAYrQoECCAxXBgQGOW5k/BwUQHGC+rx4SQEigIijAcX+gIiQwgCDH80Mcrzn/e+V4jwCCgy5xjAu+Dwk0zw0K+OXPwYEBBAUolFKczi9kS3oWm9Oy2Jxubku3HD7392tWtwYdoyPoFGUKd4eoCGpVC7bwJy58UX5hMXsyc9jlOCMuLcR7j+Vwtqjk3PPq1QwhLjKMAe0bERdZk7gGYcRHhhFVuzoBAYqTOQXcMW0N981exz9u6sitPZpZ+LcSrqK0duOpSiV1795dr1271uoYogJ5BcU8/NFGFm86xPCEJrw4shPVggOtjnXZTuUWsCX9NJvST7ElPYtNaVmkncw793hs/Zp0jIowZ9tREbSPiiAsVD7jioodzz7L7sycMmfC5pZ2Mo/SX8EBCprWrUFcZBhxkTWJbxDm+HMYdWpWPKKTW1DEvbPW801KJg/1b8Xv+8ajlHLz30xcLqXUOq1194s+JgVaVEbGqTzumbmWrRmneXRgG37bp4VP/xI4kVNgzrDTTjm+ZpGRlQ+AUtCifk06Rdc+V7jbNalFjRAp2v6ouESTdjLXcX0457xCfDK38NzzSoelzxXgBqYYx9SredkfdAuLS3j4w418mpzBpKtj+POQdgQE+O7/n75ACrRwiXX7T/DbmevILyxhym0JXN+modWRLJF55uy5M2wzPH6KI6fPAuYsqGWDcDqUnmlHR9CucS2fGGEQRl5BMXuO/XJNuHTC1p5jORSUGZauHxZCC8cZsCnGNYkrMyztLiUlmr8v2c57q/YytHMT/jm6MyFBsmDHrqRAi8s2f81Bnvx0M1G1q/O/27sT3yDc6ki2cuR0PpvTstiUnuUo3qc4lm0m8wQGKFo1DKdjVC06RtemU1QEbRqHExokRduutNYczykwM6UdZ8Slw9Ppp3657FE6LB0fGUZcg1+GplvUd25Y2p35//vNHl5ctoPeLevz3/HdqCmXYy5LQVEJ/1yxk3E9mtOsXg2Xva8UaFFlRcUl/O3z7Uz7YR+9W9bnjdu6ElFDJktVRGvN4dP55iy7zES00hm4wYGK1o3C6RhV+9w17VYNw+VMx0Jaa5ZsPsy0H/aSejSbUxcMS8eddzZsvjavV8PWoyPz1xzksY830TEqgqmTrpAlhVWUcSqP389Zz4YDp3hmaDvu6BXrsveWAi2q5FRuAffP2cCqXce465pYHh/UhqBAKSBVpbUm/VTeBWfaWWTlmUIQEhhA28bhjtnjtekQFUHLhmEEy8/c7XYePsOzC7fy457jxEXWpGeLeucNTTeJcO+wtDut2HaE++esJ6p2dWbc1YPoOq47+/MH36Vm8sC8ZAqKSnhpVCdu7NjYpe8vBVpUWuqRM/xmxloOncrnbzd1YEz3plZH8klaaw6eyGNT+qlfzrTTsjhztgiA0KAA2jWpRaeoCHq2qMeA9o0I9NJCYUdZuYW89kUKM1fvJ7xaEP/XvzVjezTzuZ/xz3tPcNf0NdQICWTGnT1p3UguUVWkpETzxte7eO2LFFo2COOt8d2Iiwxz+XGkQItK+WLbER78IJlqwYG8PaEb3ZrXsTqSXykp0ew/kcumtF+K9pb0LHIKimnTKJxHBramb+sGPj173t2KSzQfrj3IS0k7OZVbwNiezfi/G1pbet3Y3XYcPs3E934mv7CYqZOuoHtMXasj2dbJnAL+OD+ZlTszualLFH+/qYPbVmdIgRZOKdt8pEOTCN6Z2I3GEdWtjiUwBWXZlsO8nLSDfcdz6RFTl0cHtZEPT1Wwbv9Jnl24lc3pWVwRU4dnhranQ1SE1bE84uCJXCZO/ZmMU3n8Z1xX+rX1z5UY5dl48BT3zV5P5pmz/HloO8b1bObWD8NSoEWF8gqKeWTBJhZtzGBY5ya8NMo3mo/4msLiEj5Yc5B/fZlK5pmz3NCuIY8MaE3LhjJkWZGjp/N5YdkOPl6fTsNaoTxxY1uGdW7idyMRx7PPcse0NWzNOM0LN3dktFy+AswJyqyfDvDXRduIDA/lP+O60rlpbbcfVwq0KFfZ5iMPD2jNvdfG+d0vLW+TW1DE1FV7efubPeQUFDGqWzQPJraiSW0Z8bhQQVEJ037Yy5Qvd1FQVMJdvWO5v2+8Xy87yj5bxO9mrmPVrmM8PqgNv702zupIlsotKOLJT7bwyYZ0rmsdyWtjEjx2uUMKtLgk03xkPfmFxfzr1gQZ8vIyJ3MKePPrXcz4cT8omHR1DPddFyebfTh8k5LJc4u2siczh+vbNODpIe2IrV/T6li2UFBUwp/mJ7N40yHu7h3L44Paeu1M9cuxOzObe2etI/VoNn9MbMX9feM9+nOQAi0uav6agzz16Raa1K7GuxO7yzCpF0s7mctrK1L5eEMaYaFB3HtdHHdcHUv1EP+8THHgeC5//XwbK7YdIaZeDf48tJ3fdr4rT0mJ5rlFW5n+435u7hLFi6M6+dWyviWbD/HIR5sIDlRMua0LvVtGejyDFGhxnqLiEv6+ZDvvf7+Pa+Lr88bYLnLG5SN2Hj7Dy0k7+GL7URrWCuWBfq0Y0z3ab9av5xYU8dbK3bz97R6CAhR/uL4ld14TI13byqG15o2vdvHPFSn0bR3Jm+O6+nw/+cLiEp5fsoOp3++lS7PavDm2q2WXh6RAi3PKNh+5o1cMT97Y1m9+efuTNftO8MLSHazbf5IW9Wvy8IDWDOzQyGfnFmit+XzzIf7x+XYysvIZkdCExwa1pVFENaujeY05Px3gqU8307lpbd6fdIXPfmg/nJXP/XPWs3b/SSZdHcMTN7a1tIOfFGgBwK6jZ/jN9LWkn8rj7yM6MuYKmb3py7TWfLH9KC8t20Hq0Ww6N63NowNbc3VcfaujudSOw6d5duFWVu85QbvGtXhueHuukDW+VbJsyyEmz0umed0azLirh88ts/xh1zEmz9tAbkExL4zsxLDOTayOJAVawJfbj/DAvNLmI13p1lx+gfmL4hLNx+vTeG1FChlZ+fRpFcmjA1vTvol3r/3Nyi3k1RU7mbl6P7WqB/NQ/9bc5oNdwDztx93HuWfGWsKrBTHjrh4+sTFOSYnmrW9288/lO2kRGcZ/x3e1zd9LCrQf09r8w3w5aSftm9TinQndZSmOn8ovLGbmj/t5c+UuTuUWMjyhCf93Q2uX7szjCcUlmg/WHOTlpB1k5RUyrmdz/q9/K58dkrXC1owsbp+6hqKSEt6fdAVdmnlvQ5ys3EL+ND+ZL3ccZVjnJjx/c0dbLbHzywK9aGMGcZFhtGtSyyXv543yC4t55KNNLNyYwdDOTXhpZCe/ndUrfpGVV8g73+7mvVV7KS7RjO3RjD/0a0l9L9jpaN3+EzyzcCtb0k/TI6Yuzw5r79f/j7vT/uM5TJz6M0dPn+Wt8V25rnUDqyNV2pb0LO6dvY7DWfk8NbgdE69qbrt5GH5XoIuKS+jxjy85kVNA28a1GNk1iuEJUUSG2/8XkKscysrjnhnr2JKRxUP9W3PfddJ8RJzvyOl8/vVlKh+sOUi1oAB+07sFd/dpQZiNzi5KHT2dzwtLd/DxhnQa1arGE4PbMrRTY/k37WaZZ85y+9SfSTlyhldGd2ZElyirIzlFa828NQd5ZuFW6tcM4Y1xXelq01EAvyvQYBo4LNqUwYJ1aWxMyyIwQHFdq0hGdoumX9sGPr3sYt3+k/x25jryC4t5/ZYEEtvJ+k9xabszs/nn8p0s2XyYejVDuP/6eMb2bGaL/0cKikp4//u9TPkylcJizd19YrnvOv/uAuZpp/MLuWfGWlbvOcHTQ9px1zWu2wvZHfIKinnq0y0sWJ9G75b1+detXahr401Q/LJAl5V65AwfrU/j0w3pHDl9lojqwQzt3JhR3ZrSOTrCpz6Fz197kKc+2ULj2tX4nzQfEZWw8eApXli6gx/3HKdp3er83w2tGda5iWXdpVbuPMpfFm1jz7EcEts24KnB7YiRLmCWyC8s5sF5ySzbeph7r4vjkQGtbfl7c9+xHH43ax07j5zhD9e35IF+LW0/adDvC3Sp4hLNql3HWLAujaSthzlbVEJcZE1Gdovmpi5RXr2koKi4hOeX7uC9VXvpFV+PN8d2lUkzotK01nybeowXl+5g26HTtG1ci0cGtua6VpEe+4W8/3gOf128nS+2HyG2fk3+PLQdfb3w+qevKS7RPP3ZFub8dIAx3aP5x00dbdVDYdmWwzz84UYCAxWv35LgNdfMpUBfxOn8QpZsOsSC9Wms2XcSpeCa+PqM7BrNgPaNvGoyVVZuIffPXc93qceYdHUMTw2W5iPi8pSUaBZtyuCfy1M4cCKXK1vU5dGBbdw6mze3oIg3v97Fu9/uJThQ8Yd+LbmzV6ylTSTE+bTWvPZFKlO+TCWxbUPeGNvF8l3viopLeClpJ+98u4fO0RG8Oa4r0XW8Z2WCFOgK7DuWw8fr01iwPp30U3mEhQZxY8dGjOwaTY/YurYcyiklzUeEOxUUlTD35wP8+6tUjmUXMLB9Ix4a0Jr4BmEuO4bWmkWbDvH8ku0cysrnpi5RPDaoDQ1rSRcwu5rx4z6eWbiVK5rX5d3buxNRPdiSHEdP53P/3A38vPcEE65szlND2tpi7kRlSIF2UkmJ5qe9J1iwPo0lmw+RW1BMs7o1uLlrFCO7RtO0rr0+lX214wiT5yZTLTiA/47vRnfpniTcJOdsEf/7bi/vfLub/KISxnSP5oF+rS67leb2Q6d5ZuFWft57gvZNavHcsPby79hLLN6UwR8/SCYuMozpd/bw+Aeq1XuOc/+cDeScLeL5mzt6zQzzC0mBroKcs0Us23KYBevT+HHPcbSGHrF1GdU1mhs7NbZ0KYrWmv9+s4eXknbQrnEt3pnYnShpPiI84Fj2Wd74ahezf9pPgFLc0SuWe6+NI6JG5c6gTuUW8OqKFGat3k9E9WAeHtCGW65oavsJPeJ8q1KP8duZa6lTM4QZd/agRaTrRlYuRWvN29/u4eWknTSvV4O3xnWjdSPvnQwrBfoypZ/K4xPHEPjeYzlUCw5gYPtGjOwWzdVx9T36SyW/sJhHF2zis+QMhnRqzMujOnvV9XLhGw6eyOXVFSl8mpxOrWrB3HddHLdfHVPh9cjiEs28NQd4JWknWXmFjL+yOX+6QbqAebNNaae44/01AEy7owcdo93XQjYrr5CHPtzIim1HGNyxMS+O6mTLdfuVIQXaRbTWrD9wigXr01i0MYMz+UU0jqjGTV2iGNktmjg3f3o8lJXHb2euY3O6NB8R9rAt4zQvJe1g5c5MGkdU48HElozsevHtLdfuM13Atmacpmes6QLWtrF0AfMFezKzmfDez5zKLeDtCd25pqXrN2TZmpHFfbPXk34yj8dvbMudvWJ84vefFGg3yC8sZsW2IyxYn8a3KZmUaEhoWpuR3aIZ1qlJpYf8KrL+gGk+knu2iNdv7cIN0nxE2MjqPcd5YekOkg+eIr5BGA8PaE3/dg1RSnHE0QXskw3pNI6oxhM3tmWIdAHzOUdO53P71J/ZnZnNa7ckMKST63aKmr/2IE9/uoXaNYJ5c2xXn5qnIAXazY6ezufT5HQWrEtn55EzhAQGcEO7hozsFkWflpGXveTpo3VpPPHxZhpFVON/t3enlTQfETaktSZp6xFeStrBnswcujarTa/4+kxdtZfCYs09fVpwX984aoR495CkuLSsvEJ+M30Na/ef5Llh7Zl4VcxlvV9+YTHPfLaVD9Ye5Oq4eky5rYtX9IyvDCnQHqK1ZmvGaT5al8ZnyemczC2kflgoIxKaMLJbdKWH84qKS3hh6Q7+t2ovV8eZ5iN1bNyyTggw/24/WpfG61+kcvh0PoltG/L0kLY0ryddwPxBfmEx98/ZwBfbjzC5X0v+mNiySqMlB47ncu/sdWzNOM39feP54w2tfHISoRRoCxQUlfD1zqMsWJfGVzuOUlSiade4FiO7RTM8oUmFnwIvbD7y5OC2BEvzEeFF8guLOZyVL+05/VBRcQlPfLKZ+WvTGNuzGX8d3qFSxXXFtiP8aX4yCnjtlgT6tfXdS3pSoC12IqeAhcnpLFifzub0LIICFNe1bsCoblH0bfPrjTt2HT3D3TPWkXYyl78O78CtPZpZlFwIIapGa81LSTt5a+VuBnVoxGu3JFQ4y7+ouIR/rkjhrZW76RBVi7fGdbNd/wlXkwJtIzsPn2HB+jQ+2ZBO5pmz1K4RzLDOTRjZNZpO0RGs3JnJ5LkbCA0O4K3x3bjChyZDCCH8z3ur9vLXxdu4skVd3p3YnfBqF59Am3nmLJPnbuDHPce5rUcznhnazvI2op4gBdqGiopL+M6xccfybUcoKCohpl4N9p/IleYjQgif8umGdB76cCOtG4Uz7Y4eRIaff4lvzb4T/H72erLyCvn7TR0Z1S3aoqSeJwXa5rLyCvl80yEWbkw3u/cMaS/NR4QQPmXlzqPcO2s9DWqFMvPOnjSrVwOtNe+t2svzS3fQtE513hrfze/WxkuBFkIIYbkNB05yx7Q1BAUE8ObYLkz7YR9LtxxmQPuGvDy6M7UuMfzty6RACyGEsIVdR88w8b2fycjKJzBA8djANvymd6zfNq4pr0C7dd2OUmqgUmqnUmqXUuoxdx5LCCGE/cU3COeje6/m5q5RzPlNT+7u08Jvi3NF3NbSRykVCLwJ3ACkAWuUUgu11tvcdUwhhBD216R2dV4dk2B1DNtz5xl0D2CX1nqP1roAmAcMd+PxhBBCCJ/hzgIdBRws832a477zKKXuUUqtVUqtzczMdGMcIYQQwntY3jtSa/2O1rq71rp7ZGSk1XGEEEIIW3BngU4Hmpb5PtpxnxBCCCEq4M4CvQZoqZSKVUqFALcCC914PCGEEMJnuG0Wt9a6SCl1P5AEBAJTtdZb3XU8IYQQwpe4ded0rfUSYIk7jyGEEEL4IssniQkhhBDi16RACyGEEDYkBVoIIYSwISnQQgghhA1JgRZCCCFsSAq0EEIIYUO22g9aKZUJ7HfhW9YHjrnw/cTFyc/ZM+Tn7Bnyc/Yc+VlDc631Rftc26pAu5pSau2lNsIWriM/Z8+Qn7NnyM/Zc+RnXT4Z4hZCCCFsSAq0EEIIYUO+XqDfsTqAn5Cfs2fIz9kz5OfsOfKzLodPX4MWQgghvJWvn0ELIYQQXsknC7RSaqBSaqdSapdS6jGr8/gqpVRTpdTXSqltSqmtSqkHrM7ky5RSgUqpDUqpxVZn8VVKqdpKqY+UUjuUUtuVUldZnckXKaX+6PidsUUpNVcpVc3qTHbkcwVaKRUIvAkMAtoBtyml2lmbymcVAf+ntW4HXAn8Xn7WbvUAsN3qED7uX8AyrXUboDPy83Y5pVQUMBnorrXuAAQCt1qbyp58rkADPYBdWus9WusCYB4w3OJMPklrfUhrvd7x5zOYX2ZR1qbyTUqpaGAw8D+rs/gqpVQE0Ad4D0BrXaC1PmVpKN8VBFRXSgUBNYAMi/PYki8W6CjgYJnv05Ci4XZKqRigC/CTxVF81evAI0CJxTl8WSyQCbzvuJTwP6VUTatD+RqtdTrwCnAAOARkaa2XW5vKnnyxQAsPU0qFAQuAB7XWp63O42uUUkOAo1rrdVZn8XFBQFfgLa11FyAHkDksLqaUqoMZ1YwFmgA1lVLjrU1lT75YoNOBpmW+j3bcJ9xAKRWMKc6ztdYfW53HR/UChiml9mEu2VyvlJplbSSflAakaa1LR4E+whRs4VqJwF6tdabWuhD4GLja4ky25IsFeg3QUikVq5QKwUw+WGhxJp+klFKY63XbtdavWp3HV2mtH9daR2utYzD/nr/SWssZh4tprQ8DB5VSrR139QO2WRjJVx0ArlRK1XD8DumHTMa7qCCrA7ia1rpIKXU/kISZHThVa73V4li+qhcwAdislEp23PeE1nqJdZGEuCx/AGY7PtzvAe6wOI/P0Vr/pJT6CFiPWQmyAekodlHSSUwIIYSwIV8c4hZCCCG8nhRoIYQQwoakQAshhBA2JAVaCCGEsCEp0EIIIYQNSYEWQgghbEgKtBAWUUoVK6WSy9xiynnuNKXUqIvcf52z208qpWKUUnll1qyjlJqqlDqqlNpSzutGVHaXMqVUH6XUeqVU0YW5lVK3K6VSHbfby9z/tVIqWynVvTLHEsJXSYEWwjp5WuuEMrd9Hjjmbq11QpnvpwEDK3jNCMzWrZVxAJgEzCl7p1KqLvAM0BOz89wzjt7MaK37AmsreRwhfJYUaCFsRCmVoJRarZTapJT6pLR4XfCcgUqpHUqp9cDNZe6/tszZ+AalVHhFx9NafwucKCfP1cAw4GXH+8Y5k1FrvU9rvYlf7741AFihtT6htT4JrKDiDwhC+CUp0EJYp3qZgvqJ474ZwKNa607AZszZ5jlKqWrAu8BQoBvQqMzDDwG/d5wh9wbyLjeg1voHTC/7hx1n+bsrylgB2Q5WCCdJgRbCOmWHuG9SSkUAtbXW3zgenw70ueA1bTA7AaVq06e37K5W3wOvKqUmO96nyNWBncwohHABKdBC+Ait9QvAb4DqwPdKqTYWR7oY2Q5WCCdJgRbCJrTWWcBJpVRvx10TgG8ueNoOIEYpFef4/rbSB5RScVrrzVrrFzHbrrqqQJ8BwiuRsTxJQH+lVB3Htev+jvuEEBeQAi2EvdyOmZC1CUgA/lL2Qa11PnAP8LljktjRMg8/qJTa4nhtIbC0ooMppeYCPwKtlVJpSqm7LvK0ecDDjolncRVldLzvFUqpNGA08LZSaqsj/wngr5gPEGuAvzjuE0JcQLabFMJPONZZL9Zad7A6y6UopVYCD2mtZbmV8HtyBi2E/ygGIso2KrETpdTXQAvM2b8Qfk/OoIUQQggbkjNoIYQQwoakQAshhBA2JAVaCCGEsCEp0EIIIYQNSYEWQgghbOj/AVKxgRmgkV69AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(8, 6))\n", "plt.plot(acc, label = 'R-squared')\n", "plt.legend()\n", "plt.plot(-mae, label = 'MAE')\n", "plt.legend(prop={'size': 12}, loc=9)\n", "plt.xlabel('Folds [1 to 10]')\n", "plt.ylabel('Metric score [i.e., R-squared 0 to 1]')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "We can also look at the **overall accuracy** of the model" ] }, { "cell_type": "code", "execution_count": 85, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "R2: 0.5544107024745264\n", "MAE: 3.7082530497804926\n" ] } ], "source": [ "from sklearn.metrics import r2_score\n", "\n", "overall_acc = r2_score(y_train, y_pred)\n", "overall_mae = mean_absolute_error(y_train,y_pred)\n", "\n", "print('R2:', overall_acc)\n", "print('MAE:', overall_mae)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Now, let's look at the final overall model prediction" ] }, { "cell_type": "code", "execution_count": 86, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "Text(0, 0.5, 'Predicted Age')" ] }, "execution_count": 86, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA4v0lEQVR4nO3de3jcdZnw//c9x2SSNIemSUPbNOFQsB5ArLIIW7rBdTcuz4I8Xj66B1gE0R9bV9HNLrrXhZqqF1qFda2ri5R9cFdFXfFZBOIJKZWFxS0KKAUKNCltSZO0TdNkJpnj/ftjDkzSHGaSOWbu13X1ajIz+X4/00k/9/f7Ody3qCrGGGMql6PYDTDGGFNcFgiMMabCWSAwxpgKZ4HAGGMqnAUCY4ypcK5iNyAbzc3N2tHRUexmGGNMWXniiSeOquqquZ4vq0DQ0dHBnj17it0MY4wpKyJyYL7nbWjIGGMqnAUCY4ypcBYIjDGmwlkgMMaYCmeBwBhjKpwFAmNMWenr66Orq4vOzk66urro6+srdpPKngUCY0zZ6OvrY+vWrQwODtLU1MTg4CBbt261YLBEFgiMMWVj+/bteDweampqEBFqamrweDxs37692E0ra3kPBCJSJSK/EpGnROQZEfl04vFOEXlcRF4Uke+KiCffbTHGlLf+/n58Pt+0x3w+HwMDA8Vp0DJRiDuCINClqucC5wF/LCK/B3weuE1VzwRGgWsL0BZjTBnr7OwkEAhMeywQCLDcU8/ku4BY3gOBxk0kvnUn/ijQBfxH4vG7gCvy3RZjTHnr6ekhFArh9/tRVfx+P6FQiJ6enmI3LS8CoQhHxqYYDYTzep6CzBGIiFNEngSGgZ8BLwEnVDWSeMkhYM0cP3u9iOwRkT0jIyOFaK4xpkR1d3ezY8cO2traGB0dpa2tjR07dtDd3V3spuVMNKaMBcIcPB7gyNgUgVBk4R9aooIknVPVKHCeiDQAPwTOyeJnbwduB9i0aZMVWDamwnV3dy+rjj9pKhzl5FQYfzCa96GgmQqafVRVT4jIQ8CFQIOIuBJ3BWuBw4VsizHGFJuqMhGMcHIqQjAcLVo7CrFqaFXiTgARqQb+EHgWeAh4V+JlVwP/me+2GGNMKQhHYxybCPLy8QAj48GiBgEozB1BG3CXiDiJB57vqep9IrIXuFtEPgP8BthZgLYYY0zR+IMRxqciBRn3z0beA4GqPg28cZbH9wNvyff5jTGmmKIxZXwqzPhUhHA0VuzmzKqsKpQZY0y5KObkb7YsEBhjTI6oKuPBCCcnw4QipXn1PxsLBMYYs0ShSIyTU2EmpiLESvzqfzYWCIwxZpH8wQgnp8JMhoq76mepLBAYY0wWItEY41Px1T+RWPkM/8zHAoExxmRgKhzl5GQYf6j0J3+zZYHAGGPmEIu9Ovlbqks/c8ECgTHGzFBOSz9zwQKBMcYQv/qfCJXf0s9csEBgjKlo5b70MxesZrExJq/6+vro6uqis7OTrq6ukik0PxGM8MqJSQ6NBjg5Ga7YIAAWCIwxedTX18fWrVsZHBykqamJwcFBtm7dWrRgEIspJwIhXj4WYPjkFFNFzvpZKiwQGGPyZvv27Xg8HmpqahARampq8Hg8bN++vaDtCEaijIwHOXA8wHF/aNms/88VCwTGmLzp7+/H5/NNe8zn8zEwMJD3cz/wwANsedsf0bHxjVz6p+/m/p8+WBErgBbDAoExJm86OzsJBALTHgsEAnR0dOTtnJFojO/f+wBbb/o0gycCNKxcxcjwML29vezevTtv582HyXCUXc8P83f/8RT/+WT+ijhaIDBmAaU62VkOenp6CIVC+P1+VBW/308oFKKnpyfn55oMRRk6OcXLxwN8+es7cXs8+Hw+RASfz4fb7eaOnaVf/yrZ+X/qR89w5T8/Su99z/LzZ4f54W/KOBCIyDoReUhE9orIMyLy4cTjnxKRwyLyZOLPO/LdFmOyVWqTneWmu7ubHTt20NbWxujoKG1tbezYsSNnxedjMWVsMszB4wEGxybxB+OVvw4dPER1dfW011ZXV3P40KGcnDfXZuv8d+87SjASw+UQLj6zmSvOW5O380u+x8xEpA1oU9Vfi0gd8ARwBfBuYEJVv5jpsTZt2qR79uzJT0ONmUVXVxeDg4PU1NSkHvP7/bS1tfGLX/yiiC2rbAut/b/q6qsZGR6eNj8RCARY1dLCN++6q5BNndNkOMrj+4+xa98Ij+8/TjBtE5vLIbxpfSOXbFjFRWeuZF1TDU01nkWfS0SeUNVNcz1fiFKVg8Bg4utxEXkWyF9oMyaH+vv7aWpqmvZYoSY7zakCoQgnJxeu+XvdtdfS29tLIBCgurqayclJwuEw1117bYFaOruFOv/z1zeyJdH511W5C9augu4sFpEO4vWLHwcuAraKyFXAHuBjqjo6y89cD1wP0N7eXrjGGkN8snPmHUG+JzvNdLGYMj4Vz/ufaeK3zZs3c/PNN3PHzp0cPnSINWvXct2117J58+Y8t/ZUC3X+mzriV/5vPaOwnX+6vA8NpU4kUgs8DHxWVe8RkVbgKKDANuLDR++b7xg2NGQKLTlH4ElMPAYCAUKhUE7Huc3sQpEYY5Nh/MHyS/0Q7/yPs2vf8JzDPlvOXsVFZzRTW7Xw9XiDz1PeQ0OJRriBHwDfUtV7AFR1KO35bwD3FaItxmQjOdm5fft2BgYG6OjooKenp6SCQF9fH9u3b6e/v5/Ozs6CtS9f5y3Xql/ZjPkX68p/LoWYLBbgLuC4qn4k7fG2xPwBInIjcIGqvme+Y9kdgTHTFeuOJdfnjcaUiSyHf0pB8sr/4X0j/Pf+Y3nr/PN9R1CIQHAx8Evgt0DyX+kTwHuB84gPDQ0AH0gGhrlYIDBmumKtasrVeYORKCcnI0wEI2Wz6zeTYZ9cX/mX/dCQqj4CyCxPPZDvcxuz3BVrVdNSzquq+EPxso/lkvRtoc6/WKt9csXqERhTxoq1qmkx5y23ou/FuPIvFgsExpSxnp4etm7dCjBtrH62FA65nNzN5ryBULzzT+76LWXpY/6P7z/GVFrn70zr/C9eBp1/uoItH80FmyMw5lTJDn6+VU35mFSe77yLWftfLJl0/sUe9in7yeJcskBgzOIUalK5XNb+l0Pnn67sJ4uNMcWX70nlclj7X26dfyFZIDCmAuRjUrkc1v5b558ZCwTGlKje3l5uu+02xsfHqaur48Ybb+Tmm29e1LGymdxdSKmv/bfOP3sWCIwpQb29vWzbtg2Hw4HH4yEQCLBt2zaARQWDXKTKKOXhH+v8l8Ymi40pQY2NjQQCAVyuV6/VIpEIPp+P0dFTkvTmTTSmjE+FGZ+KlNzwz1Q4yuP9x9n1/PLv/G2y2JgKND4+jscz/T++0+lkYmKiIOfPZPhn9+7d3LFzJ4cOHmLtusKkea6kzh/A5XBQ5XZQ7Xbm9zx5PboxZlHq6upOuSOIRqPU1tbm9byZDv/s3r2b3t5e3G43DQ31qeLwN998c86DwWQ4yq8W6Pwv2bCKi85YyYrq8u783U4HVW4nVe74325nYcrKWyAwpgTdeOONbNu2jUgkgtPpJBqNEovFuPHGG0957VJ3DCeHf05OZp764Y6dO3G73alSkMkJ6Dt27sxJIMh0h2+5d/7Jjr/a46TK5cBVoI5/JgsExpSg5ITwbbfdxsTEBLW1tbOuGkrfMdzU1MTg4CBbt27NaMfwVDjKyakw/mA069U/hw4eoqGhftpjSy0OXwkTvi6HgypPovMv4BX/Qmyy2Jgylu2O4VhMmUjk/gkuIfNnrorDL/crf6dDEkM98Y7f4ypOx2+TxcYsY5nuGA5GooxPRZiYyk3qh6UUh89kwrdcO3+HSKrTr/I48LryO8mbKxYIjClj8+0YVlUmgvGr/6lwNKerfLItDp/s/B9+Pl7Jazl1/l63E19inN/rchAvylheKnpoqFi1Xo3Jlb6+Pq655hrGx8eJRCK4XC7qVtTzlX+5g7dcdAnRWPz/d/oqn/Qr+Hys8klablf+qUD68gHaT1vNjR+6gT/9k26cjtLv+Is+NCQi64BvAq3Ey1LerqpfFpEm4LtAB/FSle9W1YLtlFnKJJsxpSR5BapONw5fPY66ZvxhUkEA8r/KJ2m5df4Q//f970cepvcf/g6PA1ZWexk5+BJ/+5EPUeVaHv1FIWoWtwFtqvprEakDngCuAP6KeEH7W0TkJqBRVf9+vmPl8o6gWLVejcmlP+jqYnDkONUrVoIzfl0326RtV9elNDTUTxu2UFXGxsZ48MEHl9SGjIZ9zmrmojOby6bz97gc+Dyu+Fi/28Gll15a1v1F0e8IEgXpBxNfj4vIs8Aa4HJgS+JldwG7gHkDQS4Vq9ZrObAhs9IXicY4ORWhf3ichsYWSOvgZ1vGuXbd2lNW+UxOTrJm7dpFnX85jfnv3r2bnXd8g4MDL9Gx5jR6Pvo3/Mk73jHtNcu9vyjoZLGIdABvBB4HWhNBAuAI8aGj2X7meuB6gPb29py1pVi1XkudDZmVtqlwvOi7PxRf+7+2vT2jDn4pq3zSz71cOn8Rocrt4L8efojem27E63Kw0udj6PAB/uZDH8IhMu33vZj9RSEuzAo2WSwitcDDwGdV9R4ROaGqDWnPj6pq43zHyOXQUD5K9y0HNmRWmiaCEU5OhpmasfY/m0ng5GRnJqt8kpZT5+92Oqj2OPF5nFS5nDgckvHve7H6i1ydtyRKVYqIG7gP+Imq3pp47Hlgi6oOJuYRdqnq2fMdJ1+rhhablnc56uzspKmp6ZSx5NHRUfbv31/EllWeTFM/LKaDn8+CnX97A5ec3ZLzzj/XSeySm7mqPXPv4s3m970Y/UWuLsyKHggk/i98F/GJ4Y+kPb4dOJY2Wdykqn8337FsZ3H+2R1B8YUiMU5OhXO2+SsTGXX+G1blbcI3V8tbk5u5qj3x3bwLKfXf91xdmBV9shi4CPhL4Lci8mTisU8AtwDfE5FrgQPAuwvQFrOAXFayMtkJhCKcnIwQCEXycvyZV9xX/dX7cK/ZWLTOP91il7cmk7b5Eh1/tmv6S/33vVBzExW9oczMzobMCicWU8YT4//5LPySvOJ2eatxnfYaphpOJ9Z8BrherXmQz2GfhWS6vNXpkET6htwlbSvl3/dlNUeQKxYIzHIRisRSlb/yPfwzFY7yFx/9NCeqTiO26kxwpnXwGuMtnSsLduU/l7mS2LWsWsX37v5WYj1/ZsM9y00uApUFAmMKZKFlfqpKIBTNuO7vUiZPpxLFXB7eN8Jj+48xFU6724hFcY0dxH30BSYHfsMvfnx/1u8116bNEXjdTE2MEZr0s+Mfb+UdM9b0m+xZIDCmAOa7hf/Dt/8R41Px5G+ZFn5ZzOTpfBO+aAw51k/1iX7cx1/CEQkuKm10PiTH+R95+EH+6dYvcWCgv+SGaMqdBQJjCmC21ScTUyFa1qznrm/dnXXhl0zz/We62kdG9vHFz20raNK5ueRjnN/MrxRWDRmz7CVTECiCuryoq4qqagcHB4ezDgIwfwWwqbQavpmv9mnD55Kc7jfIVKkUZzFzs0BgTA50nH4Gg8fGqK5rTOX9mQwEFp3LZ2ZuIHW4mPC1UfXac7nynx89pfM/v70hUcZx7gnfzZs3F6zjr3Y78VrHXzYsEBizBP5ghJNTYa6+4aP09vaik5OLzuWT7rprr+XTn/kc0dp1SNtGwo2d4PIQAYjEMu78C8E6/vK34ByBiGwAvkY8SdzrROQNwJ+q6mcK0cB0NkdgSsFcqR9ykeohfbXPIy+MkL7YxyGkFXAvXufvcjiocjuoSuTssY6/9C15slhEHgZ6gH9R1TcmHvudqr4upy3NwGICgaVUNrkSjEQ5ORlhIhhZ1Lj/XOZb6lkKV/4uh4MqT3xlj3X85SkXk8U+Vf3VjDqc+dkDn2OWUtkslariD0Vnzfy5FKXc+ad3/LaqpzJkEgiOisgZxMtMIiLvIlFoptRt374dj8eTWtKX/Hv79u0WCMy8Ms38mY1S7fwzydJplrdMAsFfA7cD54jIYaAf+Iu8tipHlntVofkUakhsuQ29TYXjO3/9wWhOhn9KsfO3yV0zU8YbykSkBnCo6nh+mzS3bOcISj3FbL4UqojGcinuo6rxwi9TEYI5GP4ptc7fIcl1/I6KzddT6XIxWfzRWR4eA55Q1SeX1rzsZBsIlktHla1CBcByD7TJxG8TwQjR2NKu/oPhKI8PxHf4FrvzT23gcjnxuh14XQ5mzPGZCpOLyeJNiT8/Snx/GfA08EER+b6qfmHpzcyP7u5uduzYUbIpZvOlUENi5Tj0Fosp/lA8789SJ39L5crfIYLPEx/qqXI78LrK84p/uQ0zlpNMAsFa4HxVnQAQkU8C9wObgSeAkg0EEA8GlfbLVKhiFsUs6J2tqXCU8akI/uDS0j5n0vkn0zvU57Hz97gcVLud+Dwuqtzlf8VvK/yKK5NA0AIE074PE99cNikiwTl+xhRRoaoulXp1p2hMmZiK7/xdStGXUhj28bgc0/L1ZFuJq9TZCr/iyiQQfAt4XET+M/H9/wK+nZg83rvQD4vIncSHk4aTm9BE5FPA+4GRxMs+oaoPZNl2M4dCDYmV6tBbLjZ+FbPzF5F4x5/W+S+3jn+mchxmXE4yWjUkIpuI1x4G+C9VzXjGVkQ2AxPAN2cEgglV/WI2jbUUE2Y+ybw/mRR9mU0mnX8+hn0cInjdDqpcztTqnnIf6slWOSw8KOc5jJykoU50/HsSdwFXisinVfVPMvzZ3SLSkVFrTcGU8y91uuTGr/GpyKKGf+br/JO5fTLt/DOtKJZczhlfy2+reqD0hxmX+xxGJstHPcCfAH8G/BHwA+AeVf3RvD84/RgdwH0z7gj+CjgJ7AE+pqqjc/zs9cD1AO3t7W86cOBApqc1c1gOy2onQ1HGp8L4Q9lv/Mpl5580X0WxSy65hCq3o6Lr7mailIvIl8Mdy3wWvY9ARN4OvBd4O/AQ8F3gK6rasYhGdDA9ELQCR4mnrdgGtKnq+xY6zlKHhpbLVfBSlesvdSgSYyIYYSKLko9JC3X+57c3suXsxQ/7nFJRLBphcvwEq1tW8tDPfjLtit9+D8tPZ2cnTU1N0z5HVWV0dJT9+/cXsWWZWcrQ0I+BXwIXq2p/4mBfzkWjVHUorYHfAO7LxXHns9xv7Waar7Mpp4m55K7fxaz7z3fnn+7Qyy/TWOdDguNINIyg1Ljh5f0vnhIEKun3cLkop6XSizFfIDgfeA/wcxHZD9wN5OSeVkTaVDWZuO6dwO9ycdz5VNLytIU6m3L4pV7suv98DPvMRkTwJtbyV3ucnN5Sl9G/aSX9Hi4npT6HsVSZrhp6K/Fhov8NPAX8UFVvz+gEIt8BtgDNwBDwycT35xEfGhoAPpAWGOa0lKGh5K3dyZMnGRoaIhgMpv5DHjlyZFHHLLRMhxQWGvop1TmCxa77L1Tn73U7qXI5qE4UZHE4Zr/Sn+/ftNyHGCpZKc9hLGTJuYZmHMwBvA14TyZj+rm2lEDQ1dXFCy+8wPDwMCKCw+EgGo3icDi45557Sv4DzabzzqSzWcovda7HuBcz8RsMR/nVwCi7nh/O27CP0yFUe+K7d33u6R3/bDL5Ny3X+RlT3nIaCIptKYGgr6+PK6+8klgshsvlIhaLEYvFaGlp4ayzzlrSf8JCTP5l0oEk2/HII4+kglx1dTWtra24XK6cdDa5uptYzLLP9Cv//95/nMm0OYNcdf7JZZ3Vnvys7inVuzGzvFkgSLN69Wr8fj+hUAiv10traysrVqxY0m15of5jL3SVn2xHKBRicHCQaDTeSbpc8Wmg5uZm7rzzziW3aalXtFPheLWvTK/+M5nwvWTDKi4+s5l6X/adf7IaV6ZX/blQzkMMpjzlZEPZcrFx48ZZO7GlTJIWavKvs7OTF154gbGxMYLBIF6vl/r6es4666xp7RgZGcHlcuF0OolEIkSjUaqrq2lpaQHiHflS7lwWs+Io27H/fK/28bqd1HjiV/3FyNRZiYkQTWmbMxCISNNczwGo6vHcNye/8jHzX6ilmFu2bOGXv/wlDocDp9NJMBjkyJEjvP/975/WjmAwiNPpRERwuVxEIhHOOeccDh8+nJNli9msOMpm7D+fwz7JilxVHic+txOXlWI0Zpr57gieIL6qR4B2YDTxdQPwMtCZ78blWj6SpBVqKeauXbtYvXr1KXcEu3bt4uabb061w+v1EgqFcDqdxGIxvF4vgUCAYDBIY2Pjku9cFgqm2Vz952vYR0SocjvwuV1Ueco3P78xhZJJiolvEF8u+kDi+27gClX9QAHaN00pJp0rxTmC4eHh1POtra14PB4mJiZYu3ZtTpYtzjbGveXStzMeXLjWb/pqn7mu/BfT+SdX+NR4XFQXaKzfmHKRi1KVv1XV1y/0WCGUYiCAwkz+ZbNqaO/evam7ho0bN9LT08P27dvn/Pnk89nOHWSa8iFfnb/XHR/qydcKH2OWi1wEgp8QTzXx74mH/hzYrKp/lLNWZqhUA0EhLPbOIz04jI2N0dzcTEtLS+rnr776au66666Mj5tpqcdk5//wvhEee+lYTjr/9HX9y7E4izH5kotA0ER8N/Bm4nMGu4HeYkwWV0IgmG9PQrZ3HjODx/DwMEePHqW+vj6jO4X05aCToSjjwTCBYHTOlA/5uPK3q35jli5n+whEpEZV/Tlr2SIs50DQ19fHTTfdxN69e/F4PLS1teF2u5c035DJcNJ8cw/P7XuRiWA8389cE7+57vztqt+Y3FvyPoJEnqE7gFqgXUTOJZ4b6IbcNbOyJa/cjxw5klrtc/DgQdrb2/F4PIvekzDb0tZwOMxjjz1GZ2cnnZ2drFixgkAgkAoWKg78oQhrN7yeQ6OBWY+b62Eft9NBjdeFz676jSmKTDaU3Ua8IM29AKr6VKL8ZEXq7e3ltttuY3x8PLU7WVWXlFoiuRksEomk9gAADA0NcdZZZy16T8LMTWgOh4NwOIzX603tJThx4gQqDtRdTVVdA5PBMGFx8L73T18UlsvOP315p8/rxG3r+o0pqkxLVR6cUUpvcUVhy1xvby/btm3D4XDgcDgIBAL09/fT1taW9Qat9LmAI0eOsGbNmml7AFSViYkJnn76aWpra+nr68s6yMzchDY1NYWq0tDQAA4X1Q3NxKobcXuraGhs5PChQ6xZ+2qJxVx2/m6nIzHkE8/lU+mlGY0pJZlMFv8HcCuwA7gA+DCwSVXfk//mTVfsOYLGxkYCgQAulyvVqQI4HA7OO++8jHPuzJzEfe6556ZdsYsIqoqI4Ha7aWlpwePxZD1XkMy4mrwjCEdjuKvrqF7RQOcZ8dQUqsrY2BgPPvggsPCV/xsTnf/vL9D5i0gqeZvPY1f9xhRTLnINfRD4MrAGOAz8FKjI+YHx8XE8Hg/AtE1TyQRvmaaWmJmfqKGhgVdeeYVYLIbH4yEUCgHxhHHt7e3U19fj9/uznivo7+9nVWsbq9Z0oE4P/S8fJBwOE4q8OvE7OTnJaevW88sXji75yj99U5fPY1f9xpSLTALB2ar65+kPiMhFwH/lp0mlq66uLnVHkLxqB3A64xOcmaaWmDmJOz4+jsvlSgWUZJ6gqqoq6uvrgezyF4WjMSamIqw9+1yGjx7D54nX0W1ububw4Vdwu12MTQQY86zC1XkhwfXn8sl7n0n9fDadf/pEr9flsM7fmDKUSSD4CvGylQs9NisRuRO4DBhOK17fBHwX6CBeoezdqjqaWZOL58Ybb2Tbtm2pSd1wOAxAS0tLKr11JgnsZuYnSiaKS9YO6O/vJxwOE4lEGBsbo76+fsEgE43Fa/tOBCMEE1f0177/enp7ewkEAlRXV+Nwe2naeCHu9nOJrDqLOnfVqweIxTi9wcEVF2yYNuyze/du7ti5k0MHD7F2XXz+4O2X/kH8qt8meo1ZFuacIxCRC4G3Ah8hvnIoaQXwTlU9N6MTxFcYTQDfTAsEXwCOq+otInIT0Kiqf7/QsYo9RwCvrhqamJjA4/HQ2toKkFVqidnmCMLhMM3NzRw7dgxVTc0VzDdHEE3s8vUHI0yGZp+/f3DXw3z9Bz/nmKcVaT0HdaZd3WsM19ghPEdfIHL4d7Q01vHNu+5KPb179256e3txu91Ue5xMjY8RCozz1R1fsTTKxpSRRW8oE5FLiNcW/iDw9bSnxoEfqeoLWTSiA7gvLRA8D2xR1UERaQN2qerZCx0n34GgEJXGZp5rYGCAuro6hoaGGB8fTw0PRaPRVBrp2tpavv3tb9Pd3Z1R5x8MR/mfgVF2zTHmHxt+ieqxfjzH9+OITAGnTho7RLjqz/8Pw68cosbrRoj/nlhZRWPKTy5STKxX1QNLbEQH0wPBCVVtSHwtwGjy+1l+9nrgeoD29vY3HTiwpKbMqdglBNNLaSb3J9TX16OqHD9xgqeeeY5AMMpkePbsngt1/umrfT70/13HyPAwPp8v9ZpAIEDLqmZ++P27qfG4qHI7OP30063QujHLwEKBIJMB3jtEpCHtgI2JRHQ5ofFebc5opKq3q+omVd20atWqRZ2jr6+Prq4uOjs76erqoq+v75TXpK/kERFqampSu3oLobu7mwsvvJDTTz+dDRs2sKKhkZirCn/MzZqzz+PoeJBAKDItCIQiMR554Sifvf9ZrvzaY9x87zP84rnheBCIxfCOH+Z/dcAPPvhWtr/rDVz2hjbqfW6uu/ZawuEwgUAAjUaYHDtKeGyYT/zN9TTXeqlOrPjp7OwkEJi+uzgftRaMMcWVSSBoVtUTyW8Sk7otSzzvUGJIiMTfw0s83pySV/qDg4PTKnPNDAb9/f3TrpBh4ZU6mQSYbHzkY39LEBfjMQ/Rqgb8ESEUg+uuvTb1mmA4yiMvHOUz9z/LO//5UW6+9xkeTHT+AnD0JVx7+1jxP9+Ax/+N+7+2jaf2PDbtPG/r2sLnt93MafVVnBzs57SV9Xz1n/7xlDufnp4eQqEQfr8fVc1qQtwYk51c9yfZyGRo6Anik8MvJ75fT7xQTUarhhI/08H0oaHtwLG0yeImVf27hY6zmDmCTIutZ1uUPRdDSbGYMhmO4g/Fx/ujMU2t0knf5ft7b72YX/Uf5+F9Izw627DPugYuOXsV3/nHT3F88OApQz6rWlq4+1v/Rq3Xhc/jwuPKfKWPFVo3Jv/yPTSdizmCPwZuBx4mXqry94HrVTWj4SER+Q7xSedmYIh4Suv/B3yPeAnMA8SXjy6Y1noxgWChyl5JC30QMyeSjx07RigUyjhwJIUiMSZDUQLhCFPh2JzVvEKRWAadf8u0pZ5dXZfS0FD/6nuNRiAS5MTRI/S/9GJW/27GmMLJ9kI0W0veWayqPxaR84HfSzz0EVU9mmkDVPW9czx1aabHWIpMawrPV884PUgkh5defPFF1q9fP+0Ysw0lJa/6J8NRJkPReev4hiIx/vX+X3Lv/7zEZN1acHlTz83V+adbu24tI4OHqfG6kGgI0Rh+v5/O9e1Z/IsZYwpttkzB2WwiXao5A4GInKOqzyWCAMArib/bRaRdVX+d/+Yt3ULF1tN1d3fPehs2MyVEciJ5cHCQhoYGxsbGGBoaYnJyktraWu574AE2d/3hvKt8kpKrfR7eN8Iv9w0TigGNZ8SfjMXg+ACXvel0rn3HW+fc4ZvM33/T1mv58Ic+xGTirsY/z3s1xpSOTC9Y82W+weKPJf7+0ix/vpjnduVM8kq/ra2N0dFR2traMhp3S5+4eeyxx1K7iJPa2toIhUIMDQ1x4MABgpEYTt8Kalvb+ZuP93LP/T87ZZVPUvqEb3K1z4PPJYJALIbrxMv4XnqQ+j078T75PZ780Z2nBAGv28nKGi/tTT7a6qupr3Zz2Tvesaj3Ot97L/SklTGVqNgLMzKuUFYKCrWzeLadv6FQiI6OjlTunwm/H091DYeOjDAZjuGtqqa5uTmVj2hVS8u0XbqZjPk/cd+/sWLyFZzRYOq55EavT37yk+z8xu0c7H+BzrVt/N3ffiwvk7bF3k9hTKXK58KMpewsvnK+A6vqPUtsW9YKFQhmTtyMjY1x4MAB3B4vq9euZ/j4CcLq4Mwzz+To0aOsXr36lMnosbEx+n7ys6wmfK+6+upTN3r5/XgcMfwnjuF1OXLeOedqEtwYU7qWEgj+NfFlC/GcQ8le4A+AR1X1slw2NBOFCgStra2pztbrq6XltHWo080rR0YAxe12s2rVKlwuF4cPH6axsZHm5mYAVJxM+Fbjbj8PWjZktNonaffu3Xz84x/HPzFBeMqPIxqirjqezygUChGJRBgaGiIYDOJyuTjzzDN56qmnFv0+Z7v6T06CNzQ0pF5nu4mNKW+LXjWkqtckDvBTYKOqDia+bwP+b47bWTJ+dP8DjE9FUHcNntpVRFV55egJVq5cic8XH/5Jv2JvbGxkdGwc7/pzkdNeS7ixE1xeIgDhaEarfZLcDmBqnNDoEYJTkwBEpjxMTU3R3NzMwYMHERGcTifRaJS9e/cuqnJZ0kKT4Em2m9iY5S2TNNTrkkEgYYj4+v9lIRyNMRmOMhWKr/D5/Fd3snJNRzwLKPFaA5FIhNHRUWpra6murgbiV/7hxvVUnfV2VjZ0EnG/utRTgPPb45u8Lj6zmQafZ87zJ/P513id7PzKF4lMjhMKxhPBORwOotEo4+PjTE1NpYIAxGsWLKWwPcy+ZK2trY0DBw7g9/sXXGVljFkeMgkEDyZyC30n8f3/AX6evybll2p8XX8gNPu6/kMHD7Fy5Uq8Xi9Hjx4lFArhdnvw+ao5/ayzeXnSTXTV2bjWvhZHIp+/AGgMjg7QEHyF9192Md2Xzp2l2+VwUON1UuN1UeV2ph7fu3cvR48eRURwOByoKrFYDIfDkRoOCofDqZVIq1evXtI649mWrLndbjZu3MjKlSttN7ExFSKTDWVbReSdwObEQ7er6g/z26zc6uvr4/O3fpmBw0OsW9/Jtdddx+bNm1PPpxdfGR8/STQaSa0ASo75s3ojh5o6cbqrSHbdGosSGtxH7PBvqZ8cxOeKl3780m8fosYt087hcjjweZ38164H+fKtX5w11XUwGExv9rTaxcmU1ECqGP3IyAgbN25c9L/LXHssbr31Vuv4jakgGS0fTeQXOktVfy4iPsCpquN5b90Mi5ks7uvr433vex8T6iUSA7fbRW1tLZ/73OfYvHnz9OIr1dUcO3aMo6MnaH7N71HVeX5qzD9JY1GCrzzP5P49BAeeRCJTuN1uzjjjjNRrkstH//2b36TG66LW66La41xwaWZraysjIyOpzj9JRKiqqiIcDuNwOHA4HMRiMWKxGK95zWuWPGFsuYSMWd5ykWvo/cTrATSp6hkichbwdVUtSIqIdIsJBOeeey579+7F09iG0+0hFosRjUY544wzuPfee1NLNqtr6gg3rCfcfCbBhg4krYxjeuc/1f8kseAE8OoVu9vt5uyzE3V1VNFoiLGjw+x//plpHXpyWepcq3+6urp4+umnOX78OKqKw+FIVSlbsWIFdXV1DA8PEwwG8Xq9tLS0EIvFZl3NU8giO8aY0paLegR/DVwEnARIVCZbahrqgtm3bx9OpxOHI/5Wk1fUAwMDhCIxDoV8xM69ghNvfj/+11xGaNU58SAQi3H6CpDf3cfI3Z/g2AP/SOC5R1JBAEiN1btcLoiGkeAEjslRgqNDdK6dvrcA4pOzY2NjvPTSS4yPjxOJRAiFQqnVPz09PbhcLtxudyrIOBwOPv7xj/Pa174Wt9vNhg0beP3rX09rayuDg4MMDg6esvs309TbxhgDmQWCoKqGkt+IiIt5CsmUolgsRjAYZGpqimAkinvt66m9+Cqu/NqjsOk9hFedAy5PvIbviZdxPdtH27N3E3rkTjxHfocjHJjjwFE06McXm2Tq+CASmSLgn5hzlc2KFSsYHBxMBZBYLEYkEsHpdKYK4CTnA1wuF9XV1axcuZI3v/nN07agnzhxggMHDhAKhVizZs0pHX2xi+wYY8pLJkNDXwBOAFcBHwJuAPaq6j/kvXUzLDYN9cDBw9S89g/wnXkBVevfgMNTnXpeAD26H9fI8/hOHiQ4fpxwOMzNN9/MTTfdxOTkFIGAf9oxNRwkFpxAw0Hq6+v5zne+k9E4e2dn56yrfFwuF+vWraOjo2PeVLTJ4Z7HHnsMh8PBaaedlkp5kf66TFNvG2Mqw5LTUAN/D1wH/Bb4APAAcEdumpd/tS1rWXfFF3B4X90EprEYrrGDfPjdb+PiM5t5eo9wx85dHD52JFUMBmBiYoJYLIaIEIuE0VCAWDAQXypKfI/B+eefP2fW0pmOHDky6+PRaJSOjo4FU9EmzzNbR5/+umJnMjTGlJd5A4GIOIFnVPUc4BuFaVJuDQ/sw33uKOKuIvjKc0z1/5rQwaepr3Jx2WeuAmDz5s3TlnoCXHX11TQ2NHD0yGFigZNEQ1Op57xeb2r8PtuNVi6Xi2j01bQTqoqqsmXLFn7zm99w+PBhqqqqUsXrZ+vAF+ros0m9bYwx884RqGoUeF5EynYnsd/v5+h9X+LIN29k9Mf/RPCFR4n4xzh27BhXXX01u3fvPuVnvG4nh17cyyqfg7XNK+KpH9JEo9HUJG42K3E2bNgAxINB+tV8a2srd911F7W1tUB8P8HLL7/M0NDQrB34QilrF5t62xhTmTKZI9gNvBH4FZAaLFfVP13yyUUGgHEgCkTmG8OC7OcI+vr6uOyyy4jFYrjqW8Hx6i5ep9PJ+vXrU/MBl1xyCbVeF3VV8d2+s2UgfeWVV4jFYlx44YWLWo6Z3NNw8uRJwuFwalloS0tLKuPnzCI33/72t2c9j63/N8ZkKhf7CC6Z7XFVfXiJbUsGgk2Zlr7MNhB0dXXx+OOPx4vK1KxExZE8Lz6fj87OTgIT46xeWc9DP3kAh+PVq/Rs8vJns2Z/tg78hhtusMldY0zeLCUNdRXwQeBM4hPFO1U1kuPGDZDHQNDZ2YnT6eTgwYO46lsIRzW1dHP9mtXUV3sgGpqzw83kqjsXhVzyXbjaGFPZlrKh7C5gE/Eg0E28RGWuKfBTEXlCRK6f7QUicr2I7BGRPSMjI1kdvLOzE7fbTXt7O263G42GiU2exB0co8ErSCw872qa7u5ufvGLX/DVr34VgBtuuOGUzVvZrtmfrQxkscvUGWMq23yBYKOq/oWq/gvwLuD383D+i1X1fOKB5q9FZPPMF6jq7aq6SVU3rVq1KquDJztYl8vF6W3NrK7z4IhMsbKpMeMOd6Fduv39/dPqE8D0pZyZHAuwyV1jTNHMNzT060QnPev3OW+IyKeACVX94lyvWWzSufThnS1btrBr1y4GBgaoq6sD4OTJk3OO7S80bJPNsI4NARljimEpcwRRXl0lJEA1EEh8raq6YokNqwEcqjqe+PpnQK+q/niun8llqcpMx/YX2qWbzRyB7fg1xhTDoucIVNWpqisSf+pU1ZX29ZKCQEIr8IiIPEV8aer98wWBXMt0bL+zs5NAYHquofR5hWzW7C90LGOMKYZMUkzkharuB+Yu45VnC6VzSMpkl26mKSZsx68xphRlkn10Wcr06jyXu3Rtx68xphRlVKGsVBRjjmC2n7OCL8aYcpKLwjTL0mKuznt7e7n88svZtWsXBw4c4NFHH+Waa66xgi/GmLJWsYEAMtswltTX18dnPvMZwuFwKmNoMBhkaGiIyy+/fM6fM8aYUlfRgQAyK+vY19fHn/3Zn8VzFs0iEolYOUhjTNmq+ECw0DLSZKCYmJiY8xiqyuHDhwmFQlYO0hhTdoq2fLRUzLeMNHknMDExwUKT6qFQiOHh4TnvGowxplRVdCDo6+vjxIkTHDp0iOrq6mlVwerq6ti6dSt+vx+3200kEplWWWwmp9NJJBIhGAwW8B0YY8zSVezQUHLIp7a2FofDQTAY5MCBA6mqYAAej4eqqipisRhut3taaoh0DoeDWCyGquL1egv5NowxZskqNhAk5wZaW1tpb29PdeATExPs2LGDkydP4vP5aG1tRVWJRqO43W4A3G43Xq8Xp9OJiOBwOHC73bS2trJx48Zivi1jjMlaxQ4Npc8N1NfXU19fn0oA193dnSoQX19fT3t7O0NDQ0xNTVFTU8MZZ5zB0NAQY2NjNDc309LSYukijDFlq2LvCBZKMZFeLGbFihWsWbOGdevW8f3vf5+nnnqKI0eOcM8993DWWWdZughjTFmzFBPzpJiwAvHGmOVgoRQTFTk0lOzgJyYmCIVCeL1eNm7ceEpHn2lWUWOMKWcVFwj6+vq45pprGB8fJxKJ4HK58Hq9drVvjKlYFTdHcNNNN3H8+HGi0ShOp5NoNMrx48e56aabit00Y4wpioq7I9i3bx8OhwOn0wmQ+nvfvn3FbJYxxhRNUe8IROSPReR5EXlRROyS3BhjiqBogUBEnMBXgW5gI/BeEcn7bqwNGzYQjUaJxWIAxGIxotEoGzZsyPepjTGmJBXzjuAtwIuqul9VQ8DdwOX5Puktt9xCc3MzDoeDcDiMw+GgubmZW265Jd+nNsaYklTMQLAGOJj2/aHEY9OIyPUiskdE9oyMjCz5pN3d3dx5551ccMEFrF27lgsuuIA777zTVgwZYypWyU8Wq+rtwO0Q31CWi2Pa/gBjjHlVMe8IDgPr0r5fm3jMGGNMARUzEPwPcJaIdIqIB3gPcG8R22OMMRWpaENDqhoRka3ATwAncKeqPlOs9hhjTKUq6hyBqj4APFDMNhhjTKWruBQTxhhjprNAYIwxFc4CQUJfXx9dXV10dnbS1dVFX19fsZtkjDEFYYGAV4vUDA4O0tTUxODgIFu3brVgYIypCBYIeLWQfU1NDSJCTU0NHo+H7du3F7tpxhiTdxYIiBey9/l80x7z+XwMDAwUp0HGGFNAFghYuJC9McYsZxYIgJ6eHkKhEH6/H1XF7/cTCoXo6ekpdtOMMSbvLBAQT0K3Y8cO2traGB0dpa2tjR07dlhiOmNMRRDVnCT0LIhNmzbpnj17it0MY4wpKyLyhKpumuv5irsjsP0CxhgzXUUFAtsvYIwxp6qoQGD7BYwx5lQVFQhsv4AxxpyqogKB7RcwxphTVVQgsP0CxhhzqqIEAhH5lIgcFpEnE3/eUYjz2n4BY4w5VTErlN2mql8s9Em7u7ut4zfGmDQVNTRkjDHmVMUMBFtF5GkRuVNEGud6kYhcLyJ7RGTPyMhIIdtnjDEVIW8pJkTk58DqWZ76B+C/gaOAAtuANlV930LHtBQTxhiTvYVSTORtjkBV35bJ60TkG8B9+WqHMcaY+RVr1VBb2rfvBH5XjHYYY4wp3qqhL4jIecSHhgaADxSpHcYYU/GKEghU9S+LcV5jjDGnsuWjxhhT4SwQGGNMhbNAYIwxFc4CgTHGVDgLBMYYU+EsEBhjTIWzQGCMMRWuYgNBX18fXV1ddHZ20tXVZQXsjTEVqyIDQV9fH1u3bmVwcJCmpiYGBwfZunWrBQNjTEWqyECwfft2PB4PNTU1iAg1NTV4PB62b99e7KYZY0zBVWQg6O/vx+fzTXvM5/MxMDBQnAYZY0wRVWQg6OzsJBAITHssEAjQ0dFRnAYZY0wRVWQg6OnpIRQK4ff7UVX8fj+hUIienp5iN80YYwquIgNBd3c3O3bsoK2tjdHRUdra2tixY4cVtTfGVKS8larMBytVaYwx2VuoVGVF3hEYY4x5lQUCY4ypcBYIjDGmwlkgMMaYCmeBwBhjKlxZrRoSkRHgQBFO3QwcLcJ5c83eR2mx91FalvP7WK+qq+b6gbIKBMUiInvmW3pVLux9lBZ7H6Wlkt+HDQ0ZY0yFs0BgjDEVzgJBZm4vdgNyxN5HabH3UVoq9n3YHIExxlQ4uyMwxpgKZ4HAGGMqnAWCBYjIgIj8VkSeFJGySX0qIneKyLCI/C7tsSYR+ZmIvJD4u7GYbczEHO/jUyJyOPGZPCki7yhmGxciIutE5CER2Ssiz4jIhxOPl9XnMc/7KLfPo0pEfiUiTyXex6cTj3eKyOMi8qKIfFdEPMVu63zmeR//V0T60z6P8xY8ls0RzE9EBoBNqlpWG01EZDMwAXxTVV+XeOwLwHFVvUVEbgIaVfXvi9nOhczxPj4FTKjqF4vZtkyJSBvQpqq/FpE64AngCuCvKKPPY5738W7K6/MQoEZVJ0TEDTwCfBj4KHCPqt4tIl8HnlLVrxWzrfOZ5318ELhPVf8j02PZHcEypaq7geMzHr4cuCvx9V3E/xOXtDneR1lR1UFV/XXi63HgWWANZfZ5zPM+yorGTSS+dSf+KNAFJDvPcvg85nofWbNAsDAFfioiT4jI9cVuzBK1qupg4usjQGsxG7NEW0Xk6cTQUUkPqaQTkQ7gjcDjlPHnMeN9QJl9HiLiFJEngWHgZ8BLwAlVjSRecogyCHIz34eqJj+PzyY+j9tExLvQcSwQLOxiVT0f6Ab+OjFUUfY0PiZYruOCXwPOAM4DBoEvFbU1GRKRWuAHwEdU9WT6c+X0eczyPsru81DVqKqeB6wF3gKcU9wWLc7M9yEirwM+Tvz9vBloAhYcbrRAsABVPZz4exj4IfFfmnI1lBjnTY73Dhe5PYuiqkOJ/wAx4BuUwWeSGMP9AfAtVb0n8XDZfR6zvY9y/DySVPUE8BBwIdAgIq7EU2uBw8VqV7bS3scfJ4bwVFWDwL+SwedhgWAeIlKTmBRDRGqAtwO/m/+nStq9wNWJr68G/rOIbVm0ZOeZ8E5K/DNJTOrtBJ5V1VvTniqrz2Ou91GGn8cqEWlIfF0N/CHx+Y6HgHclXlYOn8ds7+O5tIsLIT7PseDnYauG5iEipxO/CwBwAd9W1c8WsUkZE5HvAFuIp6QdAj4J/D/ge0A78XTe71bVkp6IneN9bCE+DKHAAPCBtLH2kiMiFwO/BH4LxBIPf4L4+HrZfB7zvI/3Ul6fxxuITwY7iV8Mf09VexP/3+8mPpzyG+AvElfVJWme9/ELYBUgwJPAB9MmlWc/lgUCY4ypbDY0ZIwxFc4CgTHGVDgLBMYYU+EsEBhjTIWzQGCMMRXOAoExGRKRK0RERaQsd6EaMxcLBMZk7r3EMzy+t9gNMSaXLBAYk4FEfp2LgWuB9yQec4jIP4vIc4l6Ag+IyLsSz71JRB5OJCv8yYzdt8aUFAsExmTmcuDHqroPOCYibwKuBDqAjcBfEs9Xk8zH8xXgXar6JuBOoCx2pJvK5Fr4JcYY4sNBX058fXfiexfw/USytSMi8lDi+bOB1wE/i6d7wUk8K6cxJckCgTELEJEm4kVLXi8iSrxjV17NQ3XKjwDPqOqFBWqiMUtiQ0PGLOxdwL+p6npV7VDVdUA/8cpp/zsxV9BKPBkewPPAKhFJDRWJyGuL0XBjMmGBwJiFvZdTr/5/AKwmXslqL/DvwK+BMVUNEQ8enxeRp4hngHxrwVprTJYs+6gxSyAitYni4SuBXwEXqeqRYrfLmGzYHIExS3NfojiIB9hmQcCUI7sjMMaYCmdzBMYYU+EsEBhjTIWzQGCMMRXOAoExxlQ4CwTGGFPh/n9jYYmjDclbkwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.regplot(x=y_train, y=y_pred, scatter_kws=dict(color='k'))\n", "plt.ylabel('Predicted Age')" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Summary\n", "\n", "Not bad, not bad at all.\n", "\n", "But **most importantly**\n", "- this is a more **accurate estimation** of our model's predictive efficacy." ] } ], "metadata": { "anaconda-cloud": {}, "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 2 }