{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `scikit-learn`__-style API__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we give an overview of using our user-friendly `scikit-learn`-style API.\n",
    "Oftentimes, existing data analysis pipelines assume `scikit-learn` API for the models,\n",
    "so it is useful to prepare an `adelie` solver obeying such an API for seamless integration.\n",
    "Our `scikit-learn`-style solver is a simple wrapper of the more generic tools provided in `adelie`.\n",
    "Consequently, it is slightly less flexible (e.g., cannot supply a user-specified GLM).\n",
    "However, it provides many commonly used functionalities \n",
    "to serve as a quick drop-in replacement for many existing solvers in `scikit-learn` with `adelie`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.datasets import (\n",
    "    load_breast_cancer,\n",
    "    load_diabetes,\n",
    "    load_digits,\n",
    ")\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "import adelie as ad\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import scipy.stats as st\n",
    "\n",
    "np.random.seed(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Regression__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the `diabetes` dataset as a regression example and perform a train-test split.\n",
    "Note that we change the storage type for $X$ to be Fortran-order (column-major) \n",
    "since it is more efficient for our solver."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((353, 10), (353,), (89, 10), (89,))"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = load_diabetes()\n",
    "X, y = data.data, data.target\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
    "X_train = np.asfortranarray(X_train)\n",
    "X_test = np.asfortranarray(X_test)\n",
    "X_train.shape, y_train.shape, X_test.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We check that the train and test responses are similar in distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgAAAAH5CAYAAAD3DYa2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoV0lEQVR4nO3de3SU9Z3H8c+QkCFcEgiXXFZCIleBcFEoRlwayjVSRFhXpawNsAdqG2shQiHdokYtQdvlYuXA2p6Ke07VXlbQ2oU2lUsWBSRARF02ARpMVhJSZElIkAGS3/7BYY4DuZDkmZkkv/frnOfIPL/fPL/v88szzifPPJPHZYwxAgAAVukQ7AIAAEDgEQAAALAQAQAAAAsRAAAAsBABAAAACxEAAACwEAEAAAALhQa7gBvV1tbq9OnT6tatm1wuV7DLAQCgzTDG6MKFC4qLi1OHDg3/jt/qAsDp06fVt2/fYJcBAECbVVJSottuu63BPq0uAHTr1k3SteIjIiKCXA0AAG1HZWWl+vbt630vbUirCwDXT/tHREQQAAAAaIZb+QidiwABALAQAQAAAAsRAAAAsFCruwYAAND+1dTU6MqVK8Euo03q2LGjQkJCWrwdAgAAIGCMMSorK9P58+eDXUqb1r17d8XExLTo7+UQAAAAAXP9zb9Pnz7q3Lkzf/CtiYwxunjxosrLyyVJsbGxzd4WAQAAEBA1NTXeN/+ePXsGu5w2Kzw8XJJUXl6uPn36NPvjAC4CBAAExPXP/Dt37hzkStq+63PYkusoCAAAgIDitH/LOTGHBAAAACxEAAAAwEJcBAgACLp1OYUBG2vplEEBG6suCQkJWrJkiZYsWRLUOggAAAA0IiUlRaNGjdL69etbvK2DBw+qS5cuLS+qhQgAAAC0kDFGNTU1Cg1t/G21d+/eAaiocVwDAABAA+bPn689e/Zow4YNcrlccrlc2rJli1wul7Zv36677rpLbrdbe/fu1cmTJzVr1ixFR0era9euGjt2rP7yl7/4bC8hIcHnTILL5dIvf/lLzZ49W507d9bAgQP1zjvv+H2/CAAAADRgw4YNSk5O1qJFi1RaWqrS0lL17dtXkrRy5UqtWbNGx44d04gRI1RVVaX77rtP7733no4cOaLp06dr5syZKi4ubnCMrKwsPfTQQzp69Kjuu+8+zZs3T+fOnfPrfjU5AOTm5mrmzJmKi4uTy+XStm3bbupz7Ngx3X///YqMjFSXLl00duzYRnceAIDWKDIyUmFhYercubNiYmIUExPj/et7zz77rKZMmaL+/fsrKipKI0eO1He+8x0NHz5cAwcO1HPPPaf+/fs3+hv9/PnzNXfuXA0YMECrV69WVVWVPvzwQ7/uV5MDQHV1tUaOHKmNGzfW2X7y5Ende++9GjJkiHbv3q2jR49q1apV6tSpU4uLBQCgNRkzZozP46qqKi1btkx33HGHunfvrq5du+rYsWON/hI8YsQI77+7dOmiiIgI79/795cmXwSYmpqq1NTUetv/5V/+Rffdd59efPFF77r+/fvX29/j8cjj8XgfV1ZWNrUkAACC4sar+ZctW6acnBz97Gc/04ABAxQeHq4HH3xQly9fbnA7HTt29HnscrlUW1vreL1f5ei3AGpra/XHP/5RP/zhDzVt2jQdOXJEiYmJyszM1AMPPFDnc7Kzs5WVleVkGXBIY9/LDfZ3aQEgUMLCwlRTU9Nov/fff1/z58/X7NmzJV07I3Dq1Ck/V9c8jl4EWF5erqqqKq1Zs0bTp0/Xn//8Z82ePVtz5szRnj176nxOZmamKioqvEtJSYmTJQEA0GIJCQk6cOCATp06pbNnz9b72/nAgQP11ltvKT8/Xx999JG+9a1v+f03+eZy/AyAJM2aNUtLly6VJI0aNUoffPCBNm/erK9//es3PcftdsvtdjtZBgCgjWntZxSXLVumtLQ0DR06VF9++aVeffXVOvutXbtWCxcu1D333KNevXppxYoVrfajbUcDQK9evRQaGqqhQ4f6rL/jjju0d+9eJ4cCACBgBg0apH379vmsmz9//k39EhIStHPnTp916enpPo9v/EjAGHPTds6fP9+sOpvC0Y8AwsLCNHbsWBUUFPisLywsVL9+/ZwcCgAAtECTzwBUVVXpxIkT3sdFRUXKz89XVFSU4uPjtXz5cj388MOaMGGCJk6cqB07dugPf/iDdu/e7WTdAACgBZocAPLy8jRx4kTv44yMDElSWlqatmzZotmzZ2vz5s3Kzs7WE088ocGDB+s//uM/dO+99zpXNQAAaJEmB4CUlJQ6P6/4qoULF2rhwoXNLgoAAPgX9wIAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQo7+JUAAAJplV3bgxpqY2eSnpKSkaNSoUVq/fr0jJcyfP1/nz5/Xtm3bHNlec3AGAAAACxEAAABowPz587Vnzx5t2LBBLpdLLpdLp06d0ieffKLU1FR17dpV0dHRevTRR3X27Fnv837/+98rKSlJ4eHh6tmzpyZPnqzq6mo988wzeu211/T22297txeMv5ZLAAAAoAEbNmxQcnKyFi1apNLSUpWWlqpbt276xje+odGjRysvL087duzQmTNn9NBDD0mSSktLNXfuXC1cuFDHjh3T7t27NWfOHBljtGzZMj300EOaPn26d3v33HNPwPeLawAAAGhAZGSkwsLC1LlzZ8XExEiSnn/+eY0ePVqrV6/29vvVr36lvn37qrCwUFVVVbp69armzJnjvRleUlKSt294eLg8Ho93e8FAAAAAoIk++ugj7dq1S127dr2p7eTJk5o6daomTZqkpKQkTZs2TVOnTtWDDz6oHj16BKHauvERAAAATVRVVaWZM2cqPz/fZzl+/LgmTJigkJAQ5eTkaPv27Ro6dKh+/vOfa/DgwSoqKgp26V4EAAAAGhEWFqaamhrv4zvvvFOffvqpEhISNGDAAJ+lS5cukiSXy6Xx48crKytLR44cUVhYmLZu3Vrn9oKBAAAAQCMSEhJ04MABnTp1SmfPnlV6errOnTunuXPn6uDBgzp58qT+9Kc/acGCBaqpqdGBAwe0evVq5eXlqbi4WG+99Zb+9re/6Y477vBu7+jRoyooKNDZs2d15cqVgO8TAQAAgEYsW7ZMISEhGjp0qHr37q3Lly/r/fffV01NjaZOnaqkpCQtWbJE3bt3V4cOHRQREaHc3Fzdd999GjRokH784x/rX//1X5WamipJWrRokQYPHqwxY8aod+/eev/99wO+Ty5jjAn4qA2orKxUZGSkKioqFBEREexyrLYup7DB9qVTBgWoEgDtwaVLl1RUVKTExER16tQp2OW0afXNZVPeQzkDAACAhQgAAABYiAAAAICFCAAAAFiIvwQYYI1dWCdxcV1bxUWTANoSzgAAAAKqtrY22CW0eU7MIWcAAAABERYWpg4dOuj06dPq3bu3wsLC5HK5gl1Wm2KM0eXLl/W3v/1NHTp0UFhYWLO3RQAAAAREhw4dlJiYqNLSUp0+fTrY5bRpnTt3Vnx8vDp0aP6JfAIAACBgwsLCFB8fr6tXrwb9b+G3VSEhIQoNDW3x2RMCAAAgoFwulzp27KiOHTsGuxSrcREgAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFmpyAMjNzdXMmTMVFxcnl8ulbdu21dv3sccek8vl0vr161tQIgAAcFqTA0B1dbVGjhypjRs3Nthv69at2r9/v+Li4ppdHAAA8I/Qpj4hNTVVqampDfb5/PPP9f3vf19/+tOfNGPGjGYXBwAA/KPJAaAxtbW1evTRR7V8+XINGzas0f4ej0cej8f7uLKy0umSAADADRwPAC+88IJCQ0P1xBNP3FL/7OxsZWVlOV2G9dblFDbYvnTKoABVAgBojRz9FsChQ4e0YcMGbdmyRS6X65aek5mZqYqKCu9SUlLiZEkAAKAOjgaA//qv/1J5ebni4+MVGhqq0NBQffbZZ3ryySeVkJBQ53PcbrciIiJ8FgAA4F+OfgTw6KOPavLkyT7rpk2bpkcffVQLFixwcigAANACTQ4AVVVVOnHihPdxUVGR8vPzFRUVpfj4ePXs2dOnf8eOHRUTE6PBgwe3vFoAAOCIJgeAvLw8TZw40fs4IyNDkpSWlqYtW7Y4VhgAAPCfJgeAlJQUGWNuuf+pU6eaOgQAAPAz7gUAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIVCg10Amm5dTmGwS2h3GpvTpVMGBagS/7uV46c97S+AunEGAAAACxEAAACwEAEAAAALEQAAALAQAQAAAAsRAAAAsBABAAAACxEAAACwEAEAAAALEQAAALAQAQAAAAsRAAAAsBABAAAACxEAAACwEAEAAAALEQAAALAQAQAAAAsRAAAAsBABAAAACxEAAACwEAEAAAALEQAAALAQAQAAAAs1OQDk5uZq5syZiouLk8vl0rZt27xtV65c0YoVK5SUlKQuXbooLi5O3/72t3X69GknawYAAC3U5ABQXV2tkSNHauPGjTe1Xbx4UYcPH9aqVat0+PBhvfXWWyooKND999/vSLEAAMAZoU19QmpqqlJTU+tsi4yMVE5Ojs+6l19+WV/72tdUXFys+Pj4m57j8Xjk8Xi8jysrK5taEgAAaKImB4CmqqiokMvlUvfu3etsz87OVlZWlr/LQJCsyylssH3plEEBqgQA8FV+vQjw0qVLWrFihebOnauIiIg6+2RmZqqiosK7lJSU+LMkAAAgP54BuHLlih566CEZY7Rp06Z6+7ndbrndbn+VAQAA6uCXAHD9zf+zzz7Tzp076/3tHwAABIfjAeD6m//x48e1a9cu9ezZ0+khAABACzU5AFRVVenEiRPex0VFRcrPz1dUVJRiY2P14IMP6vDhw3r33XdVU1OjsrIySVJUVJTCwsKcqxwAADRbkwNAXl6eJk6c6H2ckZEhSUpLS9Mzzzyjd955R5I0atQon+ft2rVLKSkpza8UAAA4pskBICUlRcaYetsbagMAAK0D9wIAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACwUGuwC0HatyykMdgkAgGbiDAAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgoSYHgNzcXM2cOVNxcXFyuVzatm2bT7sxRk899ZRiY2MVHh6uyZMn6/jx407VCwAAHNDkAFBdXa2RI0dq48aNdba/+OKLeumll7R582YdOHBAXbp00bRp03Tp0qUWFwsAAJwR2tQnpKamKjU1tc42Y4zWr1+vH//4x5o1a5Yk6d///d8VHR2tbdu26ZFHHmlZtQAAwBGOXgNQVFSksrIyTZ482bsuMjJS48aN0759++p8jsfjUWVlpc8CAAD8q8lnABpSVlYmSYqOjvZZHx0d7W27UXZ2trKyspwso81bl1PYLsZwSmO1Lp0yKECVtMytzHlb2RcEXnt5HaD1CPq3ADIzM1VRUeFdSkpKgl0SAADtnqMBICYmRpJ05swZn/Vnzpzxtt3I7XYrIiLCZwEAAP7laABITExUTEyM3nvvPe+6yspKHThwQMnJyU4OBQAAWqDJ1wBUVVXpxIkT3sdFRUXKz89XVFSU4uPjtWTJEj3//PMaOHCgEhMTtWrVKsXFxemBBx5wsm4AANACTQ4AeXl5mjhxovdxRkaGJCktLU1btmzRD3/4Q1VXV2vx4sU6f/687r33Xu3YsUOdOnVyrmoAANAiTQ4AKSkpMsbU2+5yufTss8/q2WefbVFhAADAf4L+LQAAABB4BAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEKhwS4AaKl1OYUNti+dMihAlbQfLZ3Txp5/K/i5Af7FGQAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAs5HgAqKmp0apVq5SYmKjw8HD1799fzz33nIwxTg8FAACaKdTpDb7wwgvatGmTXnvtNQ0bNkx5eXlasGCBIiMj9cQTTzg9HAAAaAbHA8AHH3ygWbNmacaMGZKkhIQEvfHGG/rwww/r7O/xeOTxeLyPKysrnS4JAADcwPEAcM899+iVV15RYWGhBg0apI8++kh79+7V2rVr6+yfnZ2trKwsp8sImnU5hcEuATdoTz+TW9mXpVMGBaASNEVr+bk58Vrg+Go/HA8AK1euVGVlpYYMGaKQkBDV1NToJz/5iebNm1dn/8zMTGVkZHgfV1ZWqm/fvk6XBQAAvsLxAPDb3/5Wv/71r/X6669r2LBhys/P15IlSxQXF6e0tLSb+rvdbrndbqfLAAAADXA8ACxfvlwrV67UI488IklKSkrSZ599puzs7DoDAAAACDzHvwZ48eJFdejgu9mQkBDV1tY6PRQAAGgmx88AzJw5Uz/5yU8UHx+vYcOG6ciRI1q7dq0WLlzo9FAAAKCZHA8AP//5z7Vq1Sp973vfU3l5ueLi4vSd73xHTz31lNNDAQCAZnI8AHTr1k3r16/X+vXrnd40AABwCPcCAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsRAAAAMBCBAAAACxEAAAAwEIEAAAALEQAAADAQgQAAAAsFBrsAtqadTmFwS6hXWkr89lW6rRNYz+XpVMGBagSoO3hDAAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIb8EgM8//1z/9E//pJ49eyo8PFxJSUnKy8vzx1AAAKAZQp3e4P/93/9p/PjxmjhxorZv367evXvr+PHj6tGjh9NDAQCAZnI8ALzwwgvq27evXn31Ve+6xMREp4cBAAAt4PhHAO+8847GjBmjf/zHf1SfPn00evRo/eIXv6i3v8fjUWVlpc8CAAD8y/EzAH/961+1adMmZWRk6Ec/+pEOHjyoJ554QmFhYUpLS7upf3Z2trKyspwuA35wd/Er9bbtj18cwEqA1mVdTmGwS2gVNaBtcfwMQG1tre68806tXr1ao0eP1uLFi7Vo0SJt3ry5zv6ZmZmqqKjwLiUlJU6XBAAAbuB4AIiNjdXQoUN91t1xxx0qLi6us7/b7VZERITPAgAA/MvxADB+/HgVFBT4rCssLFS/fv2cHgoAADST4wFg6dKl2r9/v1avXq0TJ07o9ddf1yuvvKL09HSnhwIAAM3keAAYO3astm7dqjfeeEPDhw/Xc889p/Xr12vevHlODwUAAJrJ8W8BSNI3v/lNffOb3/THpgEAgAO4FwAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFgoNdgGo393Fr9Tbtj9+cQAraVxDtUqtr972bF1OYbBLcIQT+3Er21g6ZZDf6wBaI84AAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCG/B4A1a9bI5XJpyZIl/h4KAADcIr8GgIMHD+rf/u3fNGLECH8OAwAAmshvAaCqqkrz5s3TL37xC/Xo0aPefh6PR5WVlT4LAADwr1B/bTg9PV0zZszQ5MmT9fzzz9fbLzs7W1lZWf4qo1W4u/iVetv2xy9uN2M2pLn1tLb98Ld1OYXBLuGWtJU6b0Vd+/LV4+7uG9ra43HXFjV2DC6dMihAlbRdfjkD8Oabb+rw4cPKzs5utG9mZqYqKiq8S0lJiT9KAgAAX+H4GYCSkhL94Ac/UE5Ojjp16tRof7fbLbfb7XQZAACgAY4HgEOHDqm8vFx33nmnd11NTY1yc3P18ssvy+PxKCQkxOlhAQBAEzgeACZNmqSPP/7YZ92CBQs0ZMgQrVixgjd/AABaAccDQLdu3TR8+HCfdV26dFHPnj1vWg8AAIKDvwQIAICF/PY1wK/avXt3IIYBAAC3iDMAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFQoNdAJrn7uJXgl2ClRqa9/3xiwNYyTXNPQ4aqrWxbQZjP5urtf28GrIupzDYJdwSJ+pcOmWQ38dA4zgDAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFiIAAAAgIUIAAAAWIgAAACAhQgAAABYiAAAAICFCAAAAFjI8QCQnZ2tsWPHqlu3burTp48eeOABFRQUOD0MAABoAccDwJ49e5Senq79+/crJydHV65c0dSpU1VdXe30UAAAoJlCnd7gjh07fB5v2bJFffr00aFDhzRhwgSnhwMAAM3geAC4UUVFhSQpKiqqznaPxyOPx+N9XFlZ6e+SAACwnl8DQG1trZYsWaLx48dr+PDhdfbJzs5WVlaWP8vwWpdT2GD70imD6m/clS1Jurv4izqb98cvbnZdrcndxa8EuwS/a2wfA/2zbG31NKaheptbqz+2aYuWzF2g59073q6edXeYmOn4mMHWovcdP/PrtwDS09P1ySef6M0336y3T2ZmpioqKrxLSUmJP0sCAADy4xmAxx9/XO+++65yc3N122231dvP7XbL7Xb7qwwAAFAHxwOAMUbf//73tXXrVu3evVuJiYlODwEAAFrI8QCQnp6u119/XW+//ba6deumsrIySVJkZKTCw8OdHg4AADSD49cAbNq0SRUVFUpJSVFsbKx3+c1vfuP0UAAAoJn88hEAAABo3bgXAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWCg12AQi8u4tfCXYJPppbj7/2wx/bbck2W9vPqyHBqDXQY7aln0dLNLSf++MXt2jb+/76Rd3bvVroWD3rcurfFq7hDAAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAW8lsA2LhxoxISEtSpUyeNGzdOH374ob+GAgAATeSXAPCb3/xGGRkZevrpp3X48GGNHDlS06ZNU3l5uT+GAwAATRTqj42uXbtWixYt0oIFCyRJmzdv1h//+Ef96le/0sqVK336ejweeTwe7+OKigpJUmVlpeN1XaquarC9wTGrL137z5eeOpsb2nZ9z2nJ89D6tKWfZXupVWp+vW1pDporGHPXkvH88f/R1s7p97rr2zPGNN7ZOMzj8ZiQkBCzdetWn/Xf/va3zf33339T/6efftpIYmFhYWFhYXFoKSkpafT92vEzAGfPnlVNTY2io6N91kdHR+t//ud/buqfmZmpjIwM7+Pa2lqdO3dOPXv2lMvlcrq8oKqsrFTfvn1VUlKiiIiIYJcTFMzBNczDNczDNczDNczDNS2ZB2OMLly4oLi4uEb7+uUjgKZwu91yu90+67p37x6cYgIkIiLC6oNbYg6uYx6uYR6uYR6uYR6uae48REZG3lI/xy8C7NWrl0JCQnTmzBmf9WfOnFFMTIzTwwEAgGZwPACEhYXprrvu0nvvveddV1tbq/fee0/JyclODwcAAJrBLx8BZGRkKC0tTWPGjNHXvvY1rV+/XtXV1d5vBdjK7Xbr6aefvukjD5swB9cwD9cwD9cwD9cwD9cEah5cxtzKdwWa7uWXX9ZPf/pTlZWVadSoUXrppZc0btw4fwwFAACayG8BAAAAtF7cCwAAAAsRAAAAsBABAAAACxEAAACwEAHAYc8884xcLpfPMmTIEG/7pUuXlJ6erp49e6pr1676h3/4h5v+aFJblJubq5kzZyouLk4ul0vbtm3zaTfG6KmnnlJsbKzCw8M1efJkHT9+3KfPuXPnNG/ePEVERKh79+7653/+Z1VVta2bfDQ2D/Pnz7/p+Jg+fbpPn7Y+D9nZ2Ro7dqy6deumPn366IEHHlBBQYFPn1t5HRQXF2vGjBnq3Lmz+vTpo+XLl+vq1auB3JUWuZV5SElJuel4eOyxx3z6tPV52LRpk0aMGOH9q3bJycnavn27t92GY0FqfB6CcSwQAPxg2LBhKi0t9S579+71ti1dulR/+MMf9Lvf/U579uzR6dOnNWfOnCBW64zq6mqNHDlSGzdurLP9xRdf1EsvvaTNmzfrwIED6tKli6ZNm6ZLly55+8ybN0+ffvqpcnJy9O677yo3N1eLFy8O1C44orF5kKTp06f7HB9vvPGGT3tbn4c9e/YoPT1d+/fvV05Ojq5cuaKpU6equrra26ex10FNTY1mzJihy5cv64MPPtBrr72mLVu26KmnngrGLjXLrcyDJC1atMjneHjxxRe9be1hHm677TatWbNGhw4dUl5enr7xjW9o1qxZ+vTTTyXZcSxIjc+DFIRjoeX3/8NXPf3002bkyJF1tp0/f9507NjR/O53v/OuO3bsmJFk9u3bF6AK/U+Sz90ga2trTUxMjPnpT3/qXXf+/HnjdrvNG2+8YYwx5r//+7+NJHPw4EFvn+3btxuXy2U+//zzgNXupBvnwRhj0tLSzKxZs+p9Tnuch/LyciPJ7Nmzxxhza6+D//zP/zQdOnQwZWVl3j6bNm0yERERxuPxBHYHHHLjPBhjzNe//nXzgx/8oN7ntMd5MMaYHj16mF/+8pfWHgvXXZ8HY4JzLHAGwA+OHz+uuLg43X777Zo3b56Ki4slSYcOHdKVK1c0efJkb98hQ4YoPj5e+/btC1a5fldUVKSysjKf/Y6MjNS4ceO8+71v3z51795dY8aM8faZPHmyOnTooAMHDgS8Zn/avXu3+vTpo8GDB+u73/2uvvjiC29be5yHiooKSVJUVJSkW3sd7Nu3T0lJST53FZ02bZoqKyt9fmNqS26ch+t+/etfq1evXho+fLgyMzN18eJFb1t7m4eamhq9+eabqq6uVnJysrXHwo3zcF2gj4Wg3w2wvRk3bpy2bNmiwYMHq7S0VFlZWfr7v/97ffLJJyorK1NYWNhNdzuMjo5WWVlZcAoOgOv7Vtctoq+3lZWVqU+fPj7toaGhioqKaldzM336dM2ZM0eJiYk6efKkfvSjHyk1NVX79u1TSEhIu5uH2tpaLVmyROPHj9fw4cMl6ZZeB2VlZXUeL9fb2pq65kGSvvWtb6lfv36Ki4vT0aNHtWLFChUUFOitt96S1H7m4eOPP1ZycrIuXbqkrl27auvWrRo6dKjy8/OtOhbqmwcpOMcCAcBhqamp3n+PGDFC48aNU79+/fTb3/5W4eHhQawMrcEjjzzi/XdSUpJGjBih/v37a/fu3Zo0aVIQK/OP9PR0ffLJJz7Xwdiovnn46rUdSUlJio2N1aRJk3Ty5En1798/0GX6zeDBg5Wfn6+Kigr9/ve/V1pamvbs2RPssgKuvnkYOnRoUI4FPgLws+7du2vQoEE6ceKEYmJidPnyZZ0/f96nT3u/VfL1fWvoFtExMTEqLy/3ab969arOnTvXrufm9ttvV69evXTixAlJ7WseHn/8cb377rvatWuXbrvtNu/6W3kdxMTE1Hm8XG9rS+qbh7pcv1/KV4+H9jAPYWFhGjBggO666y5lZ2dr5MiR2rBhg3XHQn3zUJdAHAsEAD+rqqrSyZMnFRsbq7vuuksdO3b0uVVyQUGBiouL2/WtkhMTExUTE+Oz35WVlTpw4IB3v5OTk3X+/HkdOnTI22fnzp2qra1t1zeR+t///V998cUXio2NldQ+5sEYo8cff1xbt27Vzp07lZiY6NN+K6+D5ORkffzxxz5hKCcnRxEREd5Tpq1dY/NQl/z8fEnyOR7a+jzUpba2Vh6Px5pjoT7X56EuATkWmnXpIOr15JNPmt27d5uioiLz/vvvm8mTJ5tevXqZ8vJyY4wxjz32mImPjzc7d+40eXl5Jjk52SQnJwe56pa7cOGCOXLkiDly5IiRZNauXWuOHDliPvvsM2OMMWvWrDHdu3c3b7/9tjl69KiZNWuWSUxMNF9++aV3G9OnTzejR482Bw4cMHv37jUDBw40c+fODdYuNUtD83DhwgWzbNkys2/fPlNUVGT+8pe/mDvvvNMMHDjQXLp0ybuNtj4P3/3ud01kZKTZvXu3KS0t9S4XL1709mnsdXD16lUzfPhwM3XqVJOfn2927NhhevfubTIzM4OxS83S2DycOHHCPPvssyYvL88UFRWZt99+29x+++1mwoQJ3m20h3lYuXKl2bNnjykqKjJHjx41K1euNC6Xy/z5z382xthxLBjT8DwE61ggADjs4YcfNrGxsSYsLMz83d/9nXn44YfNiRMnvO1ffvml+d73vmd69OhhOnfubGbPnm1KS0uDWLEzdu3aZSTdtKSlpRljrn0VcNWqVSY6Otq43W4zadIkU1BQ4LONL774wsydO9d07drVREREmAULFpgLFy4EYW+ar6F5uHjxopk6darp3bu36dixo+nXr59ZtGiRz9d6jGn781DX/ksyr776qrfPrbwOTp06ZVJTU014eLjp1auXefLJJ82VK1cCvDfN19g8FBcXmwkTJpioqCjjdrvNgAEDzPLly01FRYXPdtr6PCxcuND069fPhIWFmd69e5tJkyZ53/yNseNYMKbheQjWscDtgAEAsBDXAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGAhAgAAABYiAAAAYCECAAAAFiIAAABgIQIAAAAWIgAAAGCh/wcjVJIyKcH5fAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "f, ax = plt.subplots(1, 1, figsize=(6, 6))\n",
    "ax.hist(y_train, bins=50, alpha=0.5, label='train')\n",
    "ax.hist(y_test, bins=50, alpha=0.5, label='test')\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __Lasso__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first instantiate our `scikit-learn`-style solver class `GroupElasticNet`.\n",
    "We will use the default setting, which solves the lasso problem with Gaussian loss.\n",
    "As usual, we call `fit()` with our training data `X_train` and `y_train` to solve \n",
    "the lasso along a path of regularization values $\\lambda$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:00<00:00:00, 2392.51it/s] [dev:52.5%]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GroupElasticNet()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;GroupElasticNet<span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GroupElasticNet()</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "GroupElasticNet()"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_gaussian = ad.GroupElasticNet()\n",
    "model_gaussian.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We call `predict()` to compute the linear predictions for each $\\lambda$ (each row)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 89)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yhatmat = model_gaussian.predict(X_test)  \n",
    "yhatmat.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we compute the out-of-sample $R^2$ for each $\\lambda$ value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAAHPCAYAAABX4QCwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIs0lEQVR4nO3deXxU1f3/8fdMyA4ZliQkQEJYRQyCrOICWqlAKYpUDS4QkbqxKL+4gbaAG0GhlFYQKVVqsSwuYPt1QTHFBaGCCZQdkSaEQkKIyASSkITM/f1BMzBkJmSfyc3r+XjM42HOnHvnczND5u25555rMQzDEAAAgIlZvV0AAABAXSPwAAAA0yPwAAAA0yPwAAAA0yPwAAAA0yPwAAAA0yPwAAAA0yPwAAAA0yPwAAAA0yPwAAAA0yPwAICkoqIi3X///YqNjVVYWJiuvvpqbd682dtlAaglBB4AkHT27FnFxcVp48aNOnnypKZOnaqRI0fq9OnT3i4NQC0g8AANxNatW3XNNdcoNDRUFotF27dv93ZJteovf/mLLBaLMjIyvPL6oaGhmjFjhmJjY2W1WjVmzBgFBARo//79XqnHl9Tne/PKK6+oW7ducjgczrYzZ87I399fzZs316xZs9xu9/rrrys2NlZFRUV1XiMaJgIPGqXdu3fr3nvvVdu2bRUYGKg2bdronnvu0e7du6u9z02bNmnWrFk6efJk7RX6PyUlJbrjjjt04sQJ/f73v9fy5cvVvn37Wn8ds/n0009lsVicD39/f3Xu3FmzZs1ScXFxhdseOHBAJ06cUOfOneupWuTl5enll1/W008/Lav1/NfT2bNn9ec//1nt27fX888/r5ycnHLb3nfffSouLtaSJUvqs2Q0IAQeNDpr1qxR7969lZKSovHjx+u1117ThAkTtGHDBvXu3Vtr166t1n43bdqk5557rk4Cz8GDB3Xo0CE98cQTevDBB3XvvfeqRYsWtf46ZvPvf/9bkjR//nwtX75cixYtUocOHfTcc89p5syZHrcrLCzUvffeq+nTp8tms9VXuY3em2++qbNnz+quu+5yaW/atKkSExP11FNPyTAM7dixo9y2QUFBSkxM1Pz582UYRn2VjAaEwING5eDBgxo7dqw6duyoHTt26MUXX9SECRP0wgsvaMeOHerYsaPGjh2r//znP94u1UXZ/9E2b97cu4U0MDt27FBQUJAeffRR3XvvvXrwwQf10UcfKSoqSqtXr3a7TdloWufOnTVjxox6rrjm8vPzvV1CtS1btky33HKLgoKC3D5/xRVXSJL27t3r9vk777xThw4d0oYNG+qsRjRcBB40KnPnzlVBQYH+9Kc/KSIiwuW58PBwLVmyRPn5+XrllVec7ffdd5/i4uLK7WvWrFmyWCzO/37yySclSR06dHCeQqnMnIdt27Zp+PDhCgsLU9OmTXXTTTfpX//6l8vrDx48WJJ0xx13yGKx6IYbbqhwn6dOndLUqVMVFxenwMBARUZG6uc//7nS0tIkSYcOHdLEiRN12WWXKTg4WK1atdIdd9zhtt6y4/z+++917733ymazKSIiQr/97W9lGIYOHz6sW2+9VWFhYYqKitLvfvc7t9vv27dPd955p8LCwtSqVSs99thjOnPmzCV/P0eOHNH999+v1q1bKzAwUFdccYXefPPNS24nnRvhueKKK+Tn5+dsCwgIUJs2bWS328v1dzgcGjt2rCwWi9566y3n+3spl3oPJem9996TxWLRl19+WW77JUuWyGKxaNeuXVU65rLf7Z49e3T33XerRYsWuu666ypVc01V5pgl6YsvvlDfvn0VFBSkTp06acmSJS7/dsqkp6drx44dGjJkiMfXLPu87Nmzx+3zffr0UcuWLfX3v/+9BkcGs2ri7QKA+vR///d/iouL0/XXX+/2+UGDBikuLk4fffRRlfY7evRoff/991q5cqV+//vfKzw8XJLKhaqL7d69W9dff73CwsL01FNPyd/fX0uWLNENN9ygL7/8UgMGDNBDDz2ktm3bavbs2Xr00UfVr18/tW7dusL9Pvzww3rvvfc0efJkde/eXT/++KM2btyovXv3qnfv3tq6das2bdqkMWPGqF27dsrIyNDixYt1ww03aM+ePQoJCSm3z4SEBF1++eWaM2eOPvroI7344otq2bKllixZop/97Gd6+eWX9be//U1PPPGE+vXrp0GDBrlsf+eddyouLk7Jycn617/+pT/+8Y/66aef9Ne//tXjcRw7dkxXX321LBaLJk+erIiICH3yySeaMGGC8vLyNHXqVI/bFhcXa//+/Ro7dqxL+9GjR7Vnzx5niLzQQw89pKysLH366adq0qRyfx4r8x5K0ogRI9S0aVO988475V579erVuuKKKxQfH1+tY77jjjvUpUsXzZ49u15O51T2mLdt26Zhw4YpOjpazz33nEpLS/X888+7/XexadMmSVLv3r09vu7jjz8uyfMIT9n233zzTU0OD2ZlAI3EyZMnDUnGrbfeWmG/W265xZBk5OXlGYZhGImJiUb79u3L9Zs5c6Zx4T+huXPnGpKM9PT0Stc0atQoIyAgwDh48KCz7ejRo0azZs2MQYMGOds2bNhgSDLefffdSu3XZrMZkyZN8vh8QUFBubbNmzcbkoy//vWvLu1lx/nggw86286ePWu0a9fOsFgsxpw5c5ztP/30kxEcHGwkJiaW2/6WW25x2e/EiRMNSca///1vwzAMY9myZeV+fxMmTDCio6ON3Nxcl23HjBlj2Gw2t8dRZtu2bYYk44UXXjCOHz9uHD161Fi3bp3Rs2dPIzQ01Ni6datL/4yMDEOSERQUZISGhjofX331lcfXMIzKv4eGYRh33XWXERkZaZw9e9bZlpWVZVitVuP555+v8jGX/W7vuuuuCmusqYvfm8oe88iRI42QkBDjyJEjzrYDBw4YTZo0MS7++vnNb35jSDJOnTrltobly5cbkozIyEgjMjLSY60PPvigERwcXJ3DhMlxSguNxqlTpyRJzZo1q7Bf2fN5eXl1Wk9paak+++wzjRo1Sh07dnS2R0dH6+6779bGjRurXUPz5s317bff6ujRo26fDw4Odv53SUmJfvzxR3Xu3FnNmzd3nva62K9//Wvnf/v5+alv374yDEMTJkxwed3LLrvM7RyoSZMmufw8ZcoUSdLHH3/s9vUMw9D777+vkSNHyjAM5ebmOh9Dhw6V3W73WKsk58TW3/72t4qIiFCbNm00bNgwtWjRQhs3blTfvn1d+rdv316GYaiwsFCnT592PjyNBkpVfw8TEhKUk5OjL774wtn23nvvyeFwKCEhodrH/PDDD3us8UIOh0Nnzpyp1MPwMFJU2WMuLS3V559/rlGjRqlNmzbOfp07d9bw4cPL7ffHH39UkyZN1LRp03LPnT59Wk8//bSGDx+ucePGKScnRydOnHBbX4sWLVRYWKiCgoJK/U7QeBB40GiUBZmy4ONJZYNRZRUXFys7O9vlUVpaquPHj6ugoECXXXZZuW0uv/xyORwOHT58uMr7lc6tZbJr1y7FxMSof//+mjVrlksIKSws1IwZMxQTE6PAwECFh4crIiJCJ0+edDu3RZJiY2NdfrbZbAoKCnKevruw/aeffiq3fZcuXVx+7tSpk6xWq8d5TsePH9fJkyed860ufIwfP16S3F6eXKbsCq2PPvpI69ev19tvv60rrrhCqamptXblVVXfw2HDhslms7lMmF69erV69eqlrl27VvuYO3ToUKl6v/rqKwUHB1fq4Wn9ocoec05OjgoLC91e1l/VS/1nz56t3Nxc/eEPf1D37t0leZ7HUxbUKjv/Co0Hc3jQaNhsNkVHR7u9pPVCO3bsUNu2bRUWFibJ8x/OsnBxKZs2bdKNN97o0paenu7xSpTK8rTfuLg43Xnnnbr++uu1du1affbZZ5o7d65efvllrVmzRsOHD9eUKVO0bNkyTZ06VQMHDpTNZpPFYtGYMWNcFny70IUTfytqk1SpeSSX+kIqq+Pee+9VYmKi2z5XXnmlx+137Nih9u3b6xe/+IWzrXfv3urevbtee+01zZ0795I11rbAwECNGjVKa9eu1WuvvaZjx47pm2++0ezZsyVV/5gvHLGrSLdu3bRs2bJK9Y2Ojq5Uv9rSqlUrnT17VqdOnXL5n42DBw9q/vz5evzxx9WlSxdnmN67d6/bCdo//fSTQkJCKv07QeNB4EGj8stf/lJLly7Vxo0b3f6x/Prrr5WRkaGHHnrI2daiRQu3a+scOnTI5WdPX+A9e/bU+vXrXdqioqLk7++vkJAQt/8nvW/fPlmtVsXExHg8Fk/7LRMdHa2JEydq4sSJysnJUe/evfXSSy9p+PDheu+995SYmOhyRdWZM2fqZA2hMgcOHHAZifjhhx/kcDjcXgEnnZvw3axZM5WWllZ45Y4nO3bsUP/+/V3aLr/8cvXt21fvv/9+rQSeiIiIKr+HCQkJeuutt5SSkqK9e/fKMAwlJCQ491eTY76UqKgo3XfffTXaR2WPOTQ0VEFBQfrhhx/K9XPX1q1bN0nnQvuFoS4pKUkRERF69tlnJZ17DyXPIzzp6enOPsCFOKWFRuXJJ59UcHCwHnroIf34448uz504cUIPP/ywQkJCnJeYS+dOvdjtdpeRoaysrHILFIaGhkpSudDQokULDRkyxOURFBQkPz8/3Xzzzfr73//uclrn2LFjWrFiha677jrnKJM7nvZbWlpa7rRUZGSk2rRp41x238/Pr9wozKuvvlrpUavqWLRoUbnXk+R2Pod0rsZf/epXev/9952Xa1/o+PHjHl8rOztbOTk5io+PL/fc0KFDlZ6eXuGVPpVVnfdwyJAhatmypVavXq3Vq1erf//+ziBYk2OuL5U9Zj8/Pw0ZMkQffPCBy1yyH374QZ988km5/Q4cOFCS9N133znbPv30U/3jH//QvHnznP++mjVrppiYGI/vX1pamq655praOFSYDCM8aFS6dOmit956S/fcc4969OihCRMmqEOHDsrIyNAbb7yh3NxcrVy5Up06dXJuM2bMGD399NO67bbb9Oijj6qgoECLFy9W165dXSaQ9unTR5L07LPPasyYMfL399fIkSOdf6jdefHFF7V+/Xpdd911mjhxopo0aaIlS5aoqKjIZS2gqjh16pTatWun22+/XT179lTTpk31+eefa+vWrc4RnV/+8pdavny5bDabunfvrs2bN+vzzz9Xq1atqvWalZGenq5bbrlFw4YN0+bNm/X222/r7rvvVs+ePT1uM2fOHG3YsEEDBgzQAw88oO7du+vEiRNKS0vT559/7nHiatn8nR49epR77uabb9ZLL72kjz76qFZGAqr6Hvr7+2v06NFatWqV8vPzNW/evFo55vpU2WOeNWuWPvvsM1177bV65JFHVFpaqoULFyo+Pr7cveA6duyo+Ph4ff7557r//vtVUlKiqVOn6sYbb3SOgJXp3r2728CTmpqqEydO6NZbb62T40YD56WrwwCv2rFjh3HXXXcZ0dHRhr+/vxEVFWXcddddxs6dO932/+yzz4z4+HgjICDAuOyyy4y333673GXphmEYL7zwgtG2bVvDarVW+hL1tLQ0Y+jQoUbTpk2NkJAQ48YbbzQ2bdrk0qcql6UXFRUZTz75pNGzZ0+jWbNmRmhoqNGzZ0/jtddec/b56aefjPHjxxvh4eFG06ZNjaFDhxr79u0z2rdv73JJuWGcv/T5+PHjLu2JiYlGaGhoudcfPHiwccUVV5Tbfs+ePcbtt99uNGvWzGjRooUxefJko7Cw0NnP3WXphmEYx44dMyZNmmTExMQ436ubbrrJ+NOf/uTxd/DKK6+4XPJ+oeLiYqNZs2bGjTfe6HH7qqrMe3ih9evXG5IMi8ViHD58uNzzlT1mT+9NbXP33lT2mFNSUoyrrrrKCAgIMDp16mT8+c9/Nh5//HEjKCioXN/58+cbTZs2NQoKCoz58+cbTZo0MXbt2lWuX1JSkmGxWMpdwv70008bsbGxhsPhqPlBw3QshsFNRwDUnVmzZum5557T8ePHy13RhcZp1KhR2r17tw4cOODSbrfb1bFjR73yyisuyx1URlFRkeLi4jRt2jQ99thjtVkuTII5PACAOlNYWOjy84EDB/Txxx+7vT2KzWbTU089pblz53q8WtCTZcuWyd/fv9JrEqHxYYQHQJ1ihKdxi46O1n333aeOHTvq0KFDWrx4sYqKirRt27ZyazMBdYlJywCAOjNs2DCtXLlS2dnZCgwM1MCBAzV79mzCDuodIzwAAMD0mMMDAABMj8ADAABMj8ADAABMj0nLOnfDvqNHj6pZs2bcYRcAgAbCMAydOnVKbdq0kdVa8RgOgUfS0aNHK7xJIwAA8F2HDx9Wu3btKuxD4NG5m9FJ535hFd2sEQAA+I68vDzFxMQ4v8crQuCRnKexwsLCCDwAADQwlZmOwqRlAABgegQeAABgegQeAABgegQeAABgegQeAABgegQeAABgegQeAABgegQeAABgegQeAABgegQeAABgegQeAABgegQeAPARWfZCbTqYqyx7oce2+uzj7dd31weoLm4eeqH8fMnPr3y7n58UFOTazxOrVQoOrl7fggLJMNz3tVikkJDq9S0slBwOz3WEhlav75kzUmlp7fQNCTlXtyQVFUlnz9ZO3+Dgc79nSSoulkpKaqdvUND5z0pV+paUnOvvSWCg1KRJ1fuePXvud+FJQIDk71/1vqWl5947T/z9z/Wval+H49xnrTb6Nmly7nchnfs3UVBQO32r8u/+or7ZR3OV8WO+4lqFKsp27t94tr1QGT/mq31EM0VHtZR07gv9UGaO4lqFauMPuZr5991yGJLVIj136xWSxaqnPj7gbLuze0v93/ajrn0kzfz7bpVKKg4I0m1XtdXabUcUUHxGfnLtU7bdyF5t9M6eE86fX/lFF8lwlH/9/22X7x8kq0W67aq2+njLfySHUa5P2XbDBnTS2m1H5DCkoNJivTjy8nJ9bunZRv/491HnfpNH95C16Ixmrd1Zrk/ZzzMS+iqhf3tl2QuVceSEOjQPLPe7Lft9Z5VYlP5jgTqEhyo6yMrfCMncfyMqy4Bht9sNSYb93J/A8o9f/MJ1g5AQ9/0kwxg82LVveLjnvn37uvZt395z3+7dXft27+65b/v2rn379vXcNzzcte/gwZ77hoS49v3FLzz3vfijdfvtFfc9ffp838TEivvm5JzvO3FixX3T08/3feKJivvu2nW+78yZFffdsuV831deqbjvhg3n+y5cWHHfDz8833fZsor7vvPO+b7vvFNx32XLzvf98MOK+y5ceL7vhg0V933llfN9t2ypuO/Mmef77tpVcd8nnjjfNz294r4TJ57vm5NTcd/ExPN9T5+uuO/ttxuGYRhHTxYY3/xwvMK+hTcPNb754bhx9GSBsWrLISPfP9Bj380x8caqLYeMVVsOGR2mfWjkBod57PvvqC5G+6c/dD4Oh0V67Lu/VaxL3/2tYj32PRwW6dL331FdPPbNDQ5z6bs5Jt5j33z/QJe+KR0r+NsjOft1ePpD46PLrq2w7xVJ7xuvf/mD0WHah8a78TdV2LfPo387t99pHxrf3z62wr7H/r3X+ZE4NWVqxZ8J/kace/jQ3wjn97fdblwKIzwA4MHqrZmavubcqENGBf02H/xR45d+q/+NO2rkJfY7/f2dkkVyGBX3u8TTtaa+XscTRyVqKDUMzflkn4xKFFv2e3UY0r/+c0JdKuj7q8WbNPn+cyPi9m8z9eAl9p1lL1R6br56nClRs0uXAh9iMYzKfHzMLS8vTzabTfajRxUWFla+A6e03PfllFbV+zJcfe6/6+GU1sWnObLthUo/WaS4ti0VbQvW6i2H9Pzq7zyeQvl/Qy/X7A3pzi/P4OLyx2aVJIt01mJVUZPzw+vu+pZxWCwq8g+8ZF+rJIfVosIm5/sGlZyRxXDtUxacDIt0xj+oXN8L+5S5uG9wSZGsMlz6XLhdYcD5voElRbL+72+Pu3279D1brCYOh8dwV9bXKimgtFiWUs9/e4r8A+X437/7gLMl8nN4/ntS6B/o/BsRcLZETYxSj38uz/gHyGKxShbJ72yJmvzv75RF53Zx4Sm+0sAgTf/g3Km5QEeJRse3dvnMPHfrFbq9T4yy7YX6z+lSdWgdpmhbsFRcrOzcvHKfR+fPrVvwN+LivpX8G+H8/rbb3X9/X4DAI1XpFwbA95T9X3eH8NBzYeaCkZmyuSdl80qsFunpYd308rp9FY6wWHVu5KGm3IUCd20XfsH6WSyaPTpekvTMml0qNQz5WSwadVUbfbDtqPPnuuzj7de/uM9Twy4r9565+z26+/3X1ft4MT+LRU8Nv0wvf7LP+VlLHt1Dkir8PCaP7qGEfrHlPscX/yyV/6w3dgSeKiLwAA3HpcJNbYYZi8XzQGrZfqoTXDy1DeoaoYzcAsWFh7h8wV3YdvHPddnH269/8c+rt2ZWOZS5C0qVeR+r6+LPTHWDkrtQJLkGp+TRPTSoa8QlQ5GZgxOBp4oIPIBvuNQf5q++P37JcFNbYabsS+iVT/ZXeWSkMsHFUxsqVhtB6VLvo1Xn5hRd+PGo69GkmobrqowmXdynJiNM3g5OBJ4qIvAA3nfxSM3Ff5jLJgRf/CXk7gumtsJM2ZdAdUZG4Fuq+j5+9f3xWhlN8rXTbu76VHeEqSan5moLgaeKCDyAd5T9IQwN8NNtr22q8heFVD7c1HaYQeNl1tNu7lRnhOlilQ1OCf1ia61uAk8VEXiA2nep/8u7cETHoupdGu0u3BBm4G2+ftrN3ST62hhhkio3urpx2o219m+QwFNFBB6gaqoSZmpylVRlJgS7CzdAQ1Qfp93c9amtEabKBqeVD1ytgZ1aVep3cikEnioi8ADnVGaSYl1e8m2txNVN7iYEA41VbV0RV9URpuqemmOEx8sIPGisKroCyt0kxbq85NvPYtGaiQNVUOxgQjDgBXWxdICn+XO1hcBTRQQeNAaXmkMjXXo+QF1e8l3bfwgBeEd9zp+ryvc399ICTKoq69e4yyYON084VLuXfN/Ssw2jN4DJRNuCXf49X/yztzDCI0Z40PBVZ/TmUiM1Fa3XwSXfAHwBp7SqiMCDhqw6t1aQyo/UVPaWCIQZAL6CwFNFBB40NJdasO9Sozee5tBU9pYIAOALmMMDmEhVF+xzN8/G0/o17ubQXBxqfOX8OwDUBIEH8GGXOl3lbni2KqM3hBkAjQWntMQpLfiW6pyuunjBPlYfBtAYcEoLaKCqc7rK04J9jN4AwHkEHsCLLpyfI51f1Viq2umqnjEt6q9oAGiACDyAl1w8P+fX13Vweym5u9NVLNgHAFVD4AG8IMte6DKa4zCkP3+d7gw3ZThdBQC1g8AD1KOyU1gn8ovLjeY4JD14XUe9sTGd01UAUMsIPEA9uXhC8sWTkv0sFo2/Lk7jr4vjdBUA1DICD1BHLjUh2aLy83M8Lf4HAKgZAg9QByozIdmQ9OqYq9SqaSCjOQBQxwg8QC2ryoTkPnEtCDoAUA+s3i4AMIMse6E2Hcx1nsZyNyH519d1lJ/FIqn8KSwAQN1ihAeoIXf3u3I3msOEZADwHkZ4gBpwd/rqlXX79fTwbm5Hc6JtwRrYqRVhBwDqGSM8QA24O31Vahi6sm1zbZx2I6M5AOAjCDxANVx4R3N3p6/KQg5BBwB8A4EHqKKL5+zcdlVbfbDtqMvqyAQdAPAtBB6gCtzN2flg21G397sCAPgOAg9QBZ7m7BQUOzSwUyvvFAUAuCQCD3AJF94iokN4qMc5OwAA30XgASpw8Xyd5NE9lDy6h55Zs4s5OwDQgFgMwzAu3c3c8vLyZLPZZLfbFRYW5u1y4COy7IW6ds4/y43mbJx2oyRxyTkAeFlVvr8Z4QE88DRfJyO3gMUDAaCBIfAAF6nMGjsAgIaFwANcgDV2AMCcCDzA/7DGDgCYF4EH+B/W2AEA8yLwoFFjjR0AaBwIPGi0WGMHABoPq7cLcGfRokWKi4tTUFCQBgwYoC1btlRqu1WrVslisWjUqFF1WyAaPHfzdZ5Zs0uDukZo47QbtfKBq7Vx2o1K6Bfr3UIBALXC5wLP6tWrlZSUpJkzZyotLU09e/bU0KFDlZOTU+F2GRkZeuKJJ3T99dfXU6VoyCpaYyfaFsw6OwBgMj4XeObPn68HHnhA48ePV/fu3fX6668rJCREb775psdtSktLdc899+i5555Tx44d67FaNDRZ9kJtOpjrXGPnQszXAQDz8qk5PMXFxUpNTdX06dOdbVarVUOGDNHmzZs9bvf8888rMjJSEyZM0Ndff33J1ykqKlJRUZHz57y8vJoVjgaBNXYAoPHyqcCTm5ur0tJStW7d2qW9devW2rdvn9ttNm7cqDfeeEPbt2+v9OskJyfrueeeq0mpaGBYYwcAGjefO6VVFadOndLYsWO1dOlShYeHV3q76dOny263Ox+HDx+uwyrhCy61xg5hBwDMzadGeMLDw+Xn56djx465tB87dkxRUVHl+h88eFAZGRkaOXKks83hcEiSmjRpov3796tTp07ltgsMDFRgYGAtVw9fxho7ANC4+dQIT0BAgPr06aOUlBRnm8PhUEpKigYOHFiuf7du3bRz505t377d+bjlllt04403avv27YqJianP8uFjyiYoZ9kLFW0LVvLoHvKznJupzJwdAGhcfGqER5KSkpKUmJiovn37qn///lqwYIHy8/M1fvx4SdK4cePUtm1bJScnKygoSPHx8S7bN2/eXJLKtaNxcbeoYEK/WA3qGqGM3ALm7ABAI+NzgSchIUHHjx/XjBkzlJ2drV69emndunXOicyZmZmyWn1qYAo+pqJFBaNtwQQdAGiELIZhGJfuZm55eXmy2Wyy2+0KCwvzdjmooU0Hc3X30m/Lta984GpuAgoAJlKV72+GSmA6ZROUL8QEZQBo3Ag8MI2yScqSmKAMAHDhc3N4gOpwN0l547QbmaAMAJDECA9MwNMkZUksKggAkETggQlUdOdzAAAkAg9MgEnKAIBLIfCgwWMVZQDApTBpGQ1Slr1Q6bn56hAeqmhbMKsoAwAqROBBg+PpthGsogwA8IRTWmhQPF2RlWUv9G5hAACfRuBBg8IVWQCA6iDwoEHhiiwAQHUQeNCgcEUWAKA6mLSMBocrsgAAVUXggc+7+BJ0SVyRBQCoEgIPfJqnS9ABAKgK5vDAZ3EJOgCgthB44LO4BB0AUFsIPPBZXIIOAKgtBB74LC5BBwDUFiYtw6dxCToAoDYQeOBzLr4MnUvQAQA1ReCBT+EydABAXWAOD3wGl6EDAOoKgQc+g8vQAQB1hcADn8Fl6ACAukLggc/gMnQAQF1h0jJ8CpehAwDqAoEHPofL0AEAtY1TWvCqLHuhNh3M5UosAECdYoQHXsOaOwCA+sIID7yCNXcAAPWJwAOvYM0dAEB9IvDAK1hzBwBQnwg88ArW3AEA1CcmLcNrWHMHAFBfCDzwKtbcAQDUB05poV6x7g4AwBsY4UG9Yd0dAIC3MMKDesG6OwAAbyLwoF6w7g4AwJsIPKgXrLsDAPAmAg/qBevuAAC8iUnLqDesuwMA8BYCD+oV6+4AALyBU1oAAMD0CDyoMywyCADwFZzSQp1gkUEAgC9hhAe1jkUGAQC+hsCDWscigwAAX0PgQa1jkUEAgK8h8KDWscggAMDXMGkZdYJFBgEAvoTAgzrDIoMAAF/BKS0AAGB6BB4AAGB6BB7UGlZWBgD4KubwoFawsjIAwJf55AjPokWLFBcXp6CgIA0YMEBbtmzx2HfNmjXq27evmjdvrtDQUPXq1UvLly+vx2rBysoAAF/nc4Fn9erVSkpK0syZM5WWlqaePXtq6NChysnJcdu/ZcuWevbZZ7V582bt2LFD48eP1/jx4/Xpp5/Wc+WNFysrAwB8ncUwDOPS3erPgAED1K9fPy1cuFCS5HA4FBMToylTpmjatGmV2kfv3r01YsQIvfDCC5Xqn5eXJ5vNJrvdrrCwsGrX3lhl2Qt17Zx/uoQeP4tFG6fdyGXpAIA6U5Xvb58a4SkuLlZqaqqGDBnibLNarRoyZIg2b958ye0Nw1BKSor279+vQYMGeexXVFSkvLw8lweqj5WVAQC+zqcmLefm5qq0tFStW7d2aW/durX27dvncTu73a62bduqqKhIfn5+eu211/Tzn//cY//k5GQ999xztVY3WFkZAODbfCrwVFezZs20fft2nT59WikpKUpKSlLHjh11ww03uO0/ffp0JSUlOX/Oy8tTTExMPVVrXqysDADwVT4VeMLDw+Xn56djx465tB87dkxRUVEet7NarercubMkqVevXtq7d6+Sk5M9Bp7AwEAFBgbWWt0AAMC3+dQcnoCAAPXp00cpKSnONofDoZSUFA0cOLDS+3E4HCoqKqqLEgEAQAPkUyM8kpSUlKTExET17dtX/fv314IFC5Sfn6/x48dLksaNG6e2bdsqOTlZ0rn5OH379lWnTp1UVFSkjz/+WMuXL9fixYu9eRiml2UvVHpuvjqEh3IaCwDg83wu8CQkJOj48eOaMWOGsrOz1atXL61bt845kTkzM1NW6/mBqfz8fE2cOFH//e9/FRwcrG7duuntt99WQkKCtw7B9FhVGQDQ0PjcOjzewDo8lceaOwAAX9Fg1+GB72NVZQBAQ0TgQZV0CA+V1eLa5mexKC48xDsFAQBQCQQeVAmrKgMAGiKfm7QM38eqygCAhobAg2phVWUAQEPCKS0AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB5USpa9UJsO5irLXujtUgAAqDKu0sIlce8sAEBDxwgPKpRlL3SGHUlyGNIza3Yx0gMAaFAIPKgQ984CAJgBgQcV4t5ZAAAzIPCgQtw7CwBgBkxaxiVx7ywAQENH4EGlcO8sAEBDxiktAABgegQeAABgegQeAABgegQeAABgegQelMN9swAAZsNVWnDBfbMAAGbECA+cuG8WAMCsCDxw4r5ZAACzIvDAiftmAQDMisADJ+6bBQAwKyYtwwX3zQIAmBGBB+Vw3ywAgNlwSgsAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJhelQNPYWGhjhw5Uq599+7dtVIQ6hd3RgcANAZVCjzvvfeeunTpohEjRujKK6/Ut99+63xu7NixtV4c6tbqrZm6ds4/dffSb3XtnH9q9dZMb5cEAECdqFLgefHFF5Wamqrt27dr2bJlmjBhglasWCFJMgzjElvDl3BndABAY1KllZZLSkrUunVrSVKfPn301Vdf6bbbbtMPP/wgi8Vyia3hSyq6MzqrLAMAzKZKIzyRkZHasWOH8+eWLVtq/fr12rt3r0s7fB93RgcANCZVCjzLly9XZGSkS1tAQIBWrlypL7/8slYLQ93izugAgMbEYjD5Rnl5ebLZbLLb7QoLC/N2OfUqy17IndEBAA1SVb6/a7QOz6FDh/TZZ58pOzvb7fNHjx6tye5RD6JtwRrYqRVhBwBgatUOPCtXrlTnzp01bNgwdezYUcuXL5ckZWZmas6cORowYIBiY2NrrVAAAIDqqnbgeeGFFzRlyhTt3LlTP//5z/XII4/ot7/9rTp16qS//OUv6tu3r959993arBUAAKBaqnRZ+oUOHjyoxx57TO3bt9eiRYsUGxurb775Rjt27NDll19emzUCAADUSLVHeEpKShQcfG7eR7t27RQUFKR58+YRdgAAgM+p0aTlFStWaN++fZIkPz8/tWjRolaKAgAAqE3VDjzXX3+9Zs6cqSuuuELh4eE6c+aM/vCHP+idd97Rnj17dPbs2dqsEwAAoNqqPYenbKHBAwcOKDU1VWlpaUpLS9Nf//pXnTx5UgEBAeratSsrMAMAAK+rduAp06VLF3Xp0kVjxoxxtqWnp+u7777Ttm3barp71LIse6HSc/PVITyUtXcAAI0GKy2r8ay0vHprpvMO6VaLlDy6hxL6sVYSAKBhqreVltFwZNkLnWFHkhyG9MyaXcqyF3q3MAAA6gGBp5FIz813hp0ypYahjNwC7xQEAEA9IvA0Eh3CQ2W1uLb5WSyKCw/xTkEAANQjAk8jEW0LVvLoHvKznEs9fhaLZo+OZ+IyAKBRqPFVWmg4EvrFalDXCGXkFiguPISwAwBoNAg8jUy0LZigAwBodDilBQAATM8nA8+iRYsUFxenoKAgDRgwQFu2bPHYd+nSpbr++uvVokULtWjRQkOGDKmwPwAAaHx8LvCsXr1aSUlJmjlzptLS0tSzZ08NHTpUOTk5bvt/8cUXuuuuu7RhwwZt3rxZMTExuvnmm3XkyJF6rhwAAPgqn1tpecCAAerXr58WLlwoSXI4HIqJidGUKVM0bdq0S25fWlqqFi1aaOHChRo3blylXrOxrLQMAICZNNiVlouLi5WamqohQ4Y426xWq4YMGaLNmzdXah8FBQUqKSlRy5YtPfYpKipSXl6eywMAAJiXTwWe3NxclZaWqnXr1i7trVu3VnZ2dqX28fTTT6tNmzYuoeliycnJstlszkdMTEyN6gYAAL7NpwJPTc2ZM0erVq3S2rVrFRQU5LHf9OnTZbfbnY/Dhw/XY5UAAKC++dQ6POHh4fLz89OxY8dc2o8dO6aoqKgKt503b57mzJmjzz//XFdeeWWFfQMDAxUYGFjjegEAQMPgUyM8AQEB6tOnj1JSUpxtDodDKSkpGjhwoMftXnnlFb3wwgtat26d+vbtWx+lAgCABsSnRngkKSkpSYmJierbt6/69++vBQsWKD8/X+PHj5ckjRs3Tm3btlVycrIk6eWXX9aMGTO0YsUKxcXFOef6NG3aVE2bNvXacfiCLHuh0nPz1SE8lNWVAQCNms8FnoSEBB0/flwzZsxQdna2evXqpXXr1jknMmdmZspqPT8wtXjxYhUXF+v222932c/MmTM1a9as+izdp6zemqnpa3bKYUhWi5Q8uocS+sV6uywAALzC59bh8QazrcOTZS/UtXP+KccF76yfxaKN025kpAcAYBoNdh0e1I703HyXsCNJpYahjNwC7xQEAICXEXhMqEN4qKwW1zY/i0Vx4SHeKQgAAC8j8JhQtC1YyaN7yM9yLvX4WSyaPTqe01kAgEbL5yYto3Yk9IvVoK4RysgtUFx4CGEHANCoEXhMLNoWTNABAECc0gIAAI0AgQcAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJgegQcAAJgegcdEsuyF2nQwV1n2Qm+XAgCAT+HmoSaxemumpq/ZKYchWS1S8ugeSugX6+2yAADwCYzwmECWvdAZdiTJYUjPrNnFSA8AAP9D4DGB9Nx8Z9gpU2oYysgt8E5BAAD4GAKPCXQID5XV4trmZ7EoLjzEOwUBAOBjCDwmEG0LVvLoHvKznEs9fhaLZo+OV7Qt2MuVAQDgG5i0bBIJ/WI1qGuEMnILFBceQtgBAOACBB4TibYFE3QAAHCDU1oAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwAAMD0CDwNVJa9UJsO5irLXujtUgAA8HncLb0BWr01U9PX7JTDkKwWKXl0DyX0i/V2WQAA+CxGeBqYLHuhM+xIksOQnlmzi5EeAAAqQOBpYNJz851hp0ypYSgjt8A7BQEA0AAQeBqYDuGhslpc2/wsFsWFh3inIAAAGgACTwMTbQtW8uge8rOcSz1+Fotmj45XtC3Yy5UBAOC7mLTcACX0i9WgrhHKyC1QXHgIYQcAgEsg8DRQ0bZggg4AAJXEKS0AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6BB4AAGB6Phl4Fi1apLi4OAUFBWnAgAHasmWLx767d+/Wr371K8XFxclisWjBggX1V2g9yrIXatPBXGXZC71dCgAADY7PBZ7Vq1crKSlJM2fOVFpamnr27KmhQ4cqJyfHbf+CggJ17NhRc+bMUVRUVD1XWz9Wb83UtXP+qbuXfqtr5/xTq7dmerskAAAaFJ8LPPPnz9cDDzyg8ePHq3v37nr99dcVEhKiN998023/fv36ae7cuRozZowCAwPrudq6l2Uv1PQ1O+Uwzv3sMKRn1uxipAcAgCrwqcBTXFys1NRUDRkyxNlmtVo1ZMgQbd68udZep6ioSHl5eS4PX5Wem+8MO2VKDUMZuQXeKQgAgAbIpwJPbm6uSktL1bp1a5f21q1bKzs7u9ZeJzk5WTabzfmIiYmptX3Xtg7hobJaXNv8LBbFhYd4pyAAABognwo89WX69Omy2+3Ox+HDh71dkkfRtmAlj+4hP8u51ONnsWj26HhF24K9XBkAAA1HE28XcKHw8HD5+fnp2LFjLu3Hjh2r1QnJgYGBDWq+T0K/WA3qGqGM3ALFhYcQdgAAqCKfGuEJCAhQnz59lJKS4mxzOBxKSUnRwIEDvViZ90XbgjWwUyvCDgAA1eBTIzySlJSUpMTERPXt21f9+/fXggULlJ+fr/Hjx0uSxo0bp7Zt2yo5OVnSuYnOe/bscf73kSNHtH37djVt2lSdO3f22nEAAADf4XOBJyEhQcePH9eMGTOUnZ2tXr16ad26dc6JzJmZmbJazw9MHT16VFdddZXz53nz5mnevHkaPHiwvvjii/ouHwAA+CCLYRjGpbuZW15enmw2m+x2u8LCwrxdDgAAqISqfH/71BweAACAukDgAQAApkfgAQAApkfgAQAApkfgAQAApkfgAQAApkfgAQAApkfgAQAApkfgAQAApkfgAQAApkfg8UFZ9kJtOpirLHuht0sBAMAUfO7moY3d6q2Zmr5mpxyGZLVIyaN7KKFfrLfLAgCgQWOEx4dk2QudYUeSHIb0zJpdjPQAAFBDBB4fkp6b7ww7ZUoNQxm5Bd4pCAAAkyDw+JAO4aGyWlzb/CwWxYWHeKcgAABMgsDjQ6JtwUoe3UN+lnOpx89i0ezR8Yq2BXu5MgAAGjYmLfuYhH6xGtQ1Qhm5BYoLDyHsAABQCwg8PijaFkzQAQCgFnFKCwAAmB6BBwAAmB6BBwAAmB6BBwAAmB6BBwAAmB6BBwAAmB6BBwAAmB6BBwAAmB6BBwAAmB6Bxwdk2Qu16WCusuyF3i4FAABT4tYSXrZ6a6amr9kphyFZLVLy6B5K6Bfr7bIAADAVRni8KMte6Aw7kuQwpGfW7GKkBwCAWkbg8aL03Hxn2ClTahjKyC3wTkEAAJgUgceLOoSHympxbfOzWBQXHuKdggAAMCkCjxdF24KVPLqH/CznUo+fxaLZo+MVbQv2cmUAAJgLk5a9LKFfrAZ1jVBGboHiwkMIOwAA1AECjw+ItgUTdAAAqEOc0gIAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4AEAAKZH4KlnWfZCbTqYqyx7obdLAQCg0eBu6fVo9dZMTV+zUw5Dslqk5NE9lNAv1ttlAQBgeozw1JMse6Ez7EiSw5CeWbOLkR4AAOoBIzz1JD033xl2ypQahjJyCxRtC/ZOUQBQS0pLS1VSUuLtMmAy/v7+8vPzq5V9EXjqSYfwUFktcgk9fhaL4sJDvFcUANSQYRjKzs7WyZMnvV0KTKp58+aKioqSxWKp0X4IPPUk2has5NE99MyaXSo1DPlZLJo9Op7RHQANWlnYiYyMVEhISI2/lIAyhmGooKBAOTk5kqTo6Oga7Y/AU48S+sVqUNcIZeQWKC48hLADoEErLS11hp1WrVp5uxyYUHDwue/JnJwcRUZG1uj0FoGnnkXbggk6AEyhbM5OSAin5lF3yj5fJSUlNQo8XKUFAKgRTmOhLtXW54vAAwAATI/AAwAATM8nA8+iRYsUFxenoKAgDRgwQFu2bKmw/7vvvqtu3bopKChIPXr00Mcff1xPlQIAgIbA5wLP6tWrlZSUpJkzZyotLU09e/bU0KFDnZelXWzTpk266667NGHCBG3btk2jRo3SqFGjtGvXrnquHACAunH48GHdcMMN6t69u6688kq9++673i6pwbEYhmFculv9GTBggPr166eFCxdKkhwOh2JiYjRlyhRNmzatXP+EhATl5+frww8/dLZdffXV6tWrl15//fVKvWZeXp5sNpvsdrvCwsJq50AAwOTOnDmj9PR0dejQQUFBQd4ux9SysrJ07Ngx9erVS9nZ2erTp4++//57hYaGeru0OlfR56wq398+NcJTXFys1NRUDRkyxNlmtVo1ZMgQbd682e02mzdvdukvSUOHDvXYHwCAhiY6Olq9evWSJEVFRSk8PFwnTpzwblENjE8FntzcXJWWlqp169Yu7a1bt1Z2drbbbbKzs6vUX5KKioqUl5fn8gAAwBsGDx4si8Uii8WigIAAXX755VqxYoXH/qmpqSotLVVMTEyd1lXV+bQXmjNnjiwWi6ZOnerSHhcX5zzWCx+TJk2q5erL86nAU1+Sk5Nls9mcj7r+0AAA4I5hGNq2bZvmzZunrKws7d+/X8OGDdO4ceOUnp5erv+JEyc0btw4/elPf6rTuqo6n/ZCW7du1ZIlS3TllVe6fS4rK8v5WL9+vSTpjjvuqPVjuJhPBZ7w8HD5+fnp2LFjLu3Hjh1TVFSU222ioqKq1F+Spk+fLrvd7nwcPny45sUDABqMG264QVOmTNHUqVPVokULtW7dWkuXLlV+fr7Gjx+vZs2aqXPnzvrkk0+c26xbt07XXXedmjdvrlatWumXv/ylDh486Hz++PHjioqK0uzZs51tmzZtUkBAgFJSUtzWceDAAZ06dUrDhg1TVFSUOnTooAkTJqi0tFT79+936VtUVKRRo0Zp2rRpuuaaa2r5N+Jq/vz5euCBBzR+/Hh1795dr7/+ukJCQvTmm29WuN3p06d1zz33aOnSpWrRokW55yMiIhQVFeV8fPjhh+rUqZMGDx5cV4fi5FOBJyAgQH369HH5YDgcDqWkpGjgwIFutxk4cGC5D9L69es99pekwMBAhYWFuTwAALUoP9/z48yZyvctLLx032p66623FB4eri1btmjKlCl65JFHdMcdd+iaa65RWlqabr75Zo0dO1YFBQX/e+l8JSUl6bvvvlNKSoqsVqtuu+02ORwOSee+zN98803NmjVL3333nU6dOqWxY8dq8uTJuummm9zWkJqaqhYtWqh79+6SpP/+97969tlnFRgY6DJCYhiG7rvvPv3sZz/T2LFjL3lss2fPVtOmTSt8ZGZmut22OvNpy0yaNEkjRowoN7fW0+u8/fbbuv/+++tntW7Dx6xatcoIDAw0/vKXvxh79uwxHnzwQaN58+ZGdna2YRiGMXbsWGPatGnO/t98843RpEkTY968ecbevXuNmTNnGv7+/sbOnTsr/Zp2u92QZNjt9lo/HgAwq8LCQmPPnj1GYWFh+Sclz49f/MK1b0iI576DB7v2DQ8v36caBg8ebFx33XXOn8+ePWuEhoYaY8eOdbZlZWUZkozNmze73cfx48cNSeW+byZOnGh07drVuPvuu40ePXoYZ86c8VjHE088YVitViM0NNQICgoyJBnBwcHGsmXLXPp9/fXXhsViMXr27Ol87Nixw+N+f/zxR+PAgQMVPkpKStxue+TIEUOSsWnTJpf2J5980ujfv7/H11y5cqURHx/v/DwMHjzYeOyxxzz2X716teHn52ccOXLEYx/DqPhzVpXvb5+7eWhCQoKOHz+uGTNmKDs7W7169dK6deucE5MzMzNltZ4fmLrmmmu0YsUK/eY3v9EzzzyjLl266IMPPlB8fLy3DgEA0ABcOILi5+enVq1aqUePHs62su+dsnkrBw4c0IwZM/Ttt98qNzfXObKTmZnp8p0zb948xcfH691331VqaqoCAwM91pCWlqZJkybp0Ucf1cmTJ/XEE0/o2muv1X333efS77rrrnO+XmW0bNlSLVu2rHT/mjp8+LAee+wxrV+/vtJLFLzxxhsaPny42rRpU8fVneNzgUeSJk+erMmTJ7t97osvvijXdscdd9TLhKeqyrIXKj03Xx3CQ7lDOoDG5fRpz89dfMfriibCWi+aeZGRUe2SLubv7+/ys8VicWkrO81SFjRGjhyp9u3ba+nSpWrTpo0cDofi4+NVXFzssp+DBw/q6NGjcjgcysjIcAlRF0tLS9MDDzygzp07S5Jee+01XXnllXrggQcUFxdX7WObPXu2y1wid/bs2aPY2Nhy7dWZT5uamqqcnBz17t3b2VZaWqqvvvpKCxcuVFFRkcudzg8dOqTPP/9ca9asqcph1YhPBh4zWL01U9PX7JTDkKwWKXl0DyX0K//BAgBTqsqCeHXVtxb9+OOP2r9/v5YuXarrr79ekrRx48Zy/YqLi3XvvfcqISFBl112mX79619r586dioyMLNf3P//5j06ePOkyOtS9e3d16tRJK1as0DPPPFPteh9++GHdeeedFfbxNLJy4XzaUaNGSTo/n9bTYMRNN92knTt3urSNHz9e3bp109NPP+0SdiRp2bJlioyM1IgRIyp5RDVH4KkDWfZCZ9iRJIchPbNmlwZ1jWCkBwAaoBYtWqhVq1b605/+pOjoaGVmZrpd/f/ZZ5+V3W7XH//4RzVt2lQff/yx7r//fpe7AZRJTU2Vv7+/unbt6tJ+0003ae3atTUKPDU9pZWUlKTExET17dtX/fv314IFC5xXsJVZuHCh1q5dq5SUFDVr1qzcVJLQ0FC1atWqXLvD4dCyZcuUmJioJk3qL4b41FVaZpGem+8MO2VKDUMZuQXeKQgAUCNWq1WrVq1Samqq4uPj9f/+3//T3LlzXfp88cUXWrBggZYvX66wsDBZrVYtX75cX3/9tRYvXlxun2lpaerSpYsCAgJc2ocMGaLU1FT997//rdNjqkhCQoLmzZunGTNmqFevXtq+fbvLfFrp3GLBF16WX1mff/65MjMzdf/999dmyZfkc/fS8obavpdWlr1Q1875p0vo8bNYtHHajYzwADAN7qWF+mDKe2mZRbQtWMmje8jvfxPe/CwWzR4dT9gBAMBLmMNTRxL6xWpQ1whl5BYoLjyEsAMAgBcReOpQtC2YoAMAgA/glBYAADA9Ag8AADA9Ag8AoEa42Bd1qbY+XwQeAEC1lN2Goexu4kBdKPt8XXwrkKpi0jIAoFr8/PzUvHlz5801Q0JCnPefAmrKMAwVFBQoJydHzZs3L3d7iqoi8AAAqq3sZpI5Fd0AFKiB5s2be7xpaVUQeAAA1WaxWBQdHa3IyEiVlJR4uxyYjL+/f41HdsoQeAAANebn51drX0xAXWDSMgAAMD0CDwAAMD0CDwAAMD3m8Oj8okZ5eXlergQAAFRW2fd2ZRYnJPBIOnXqlCQpJibGy5UAAICqOnXqlGw2W4V9LAZrgsvhcOjo0aNq1qyZTp06pZiYGB0+fFhhYWHeLq3RysvL433wAbwPvoH3wft4D3zDxe+DYRg6deqU2rRpI6u14lk6jPBIslqtateunSQ5VwkNCwvjQ+0DeB98A++Db+B98D7eA99w4ftwqZGdMkxaBgAApkfgAQAApkfguUhgYKBmzpypwMBAb5fSqPE++AbeB9/A++B9vAe+oSbvA5OWAQCA6THCAwAATI/AAwAATI/AAwAATI/AAwAATI/AU4GXXnpJ11xzjUJCQtS8eXNvl9NoLFq0SHFxcQoKCtKAAQO0ZcsWb5fU6Hz11VcaOXKk2rRpI4vFog8++MDbJTU6ycnJ6tevn5o1a6bIyEiNGjVK+/fv93ZZjc7ixYt15ZVXOhe6GzhwoD755BNvl9XozZkzRxaLRVOnTq30NgSeChQXF+uOO+7QI4884u1SGo3Vq1crKSlJM2fOVFpamnr27KmhQ4cqJyfH26U1Kvn5+erZs6cWLVrk7VIarS+//FKTJk3Sv/71L61fv14lJSW6+eablZ+f7+3SGpV27dppzpw5Sk1N1Xfffaef/exnuvXWW7V7925vl9Zobd26VUuWLNGVV15Zpe24LL0S/vKXv2jq1Kk6efKkt0sxvQEDBqhfv35auHChpHP3OYuJidGUKVM0bdo0L1fXOFksFq1du1ajRo3ydimN2vHjxxUZGakvv/xSgwYN8nY5jVrLli01d+5cTZgwwdulNDqnT59W79699dprr+nFF19Ur169tGDBgkptywgPfEZxcbFSU1M1ZMgQZ5vVatWQIUO0efNmL1YGeJ/dbpd07ssW3lFaWqpVq1YpPz9fAwcO9HY5jdKkSZM0YsQIl++JyuLmofAZubm5Ki0tVevWrV3aW7durX379nmpKsD7HA6Hpk6dqmuvvVbx8fHeLqfR2blzpwYOHKgzZ86oadOmWrt2rbp37+7tshqdVatWKS0tTVu3bq3W9o1uhGfatGmyWCwVPvhyBeBLJk2apF27dmnVqlXeLqVRuuyyy7R9+3Z9++23euSRR5SYmKg9e/Z4u6xG5fDhw3rsscf0t7/9TUFBQdXaR6Mb4Xn88cd13333VdinY8eO9VMMXISHh8vPz0/Hjh1zaT927JiioqK8VBXgXZMnT9aHH36or776Su3atfN2OY1SQECAOnfuLEnq06ePtm7dqj/84Q9asmSJlytrPFJTU5WTk6PevXs720pLS/XVV19p4cKFKioqkp+fX4X7aHSBJyIiQhEREd4uA24EBASoT58+SklJcU6QdTgcSklJ0eTJk71bHFDPDMPQlClTtHbtWn3xxRfq0KGDt0vC/zgcDhUVFXm7jEblpptu0s6dO13axo8fr27duunpp5++ZNiRGmHgqYrMzEydOHFCmZmZKi0t1fbt2yVJnTt3VtOmTb1bnEklJSUpMTFRffv2Vf/+/bVgwQLl5+dr/Pjx3i6tUTl9+rR++OEH58/p6enavn27WrZsqdjYWC9W1nhMmjRJK1as0N///nc1a9ZM2dnZkiSbzabg4GAvV9d4TJ8+XcOHD1dsbKxOnTqlFStW6IsvvtCnn37q7dIalWbNmpWbvxYaGqpWrVpVfl6bAY8SExMNSeUeGzZs8HZppvbqq68asbGxRkBAgNG/f3/jX//6l7dLanQ2bNjg9rOfmJjo7dIaDXe/f0nGsmXLvF1ao3L//fcb7du3NwICAoyIiAjjpptuMj777DNvlwXDMAYPHmw89thjle7POjwAAMD0Gt1VWgAAoPEh8AAAANMj8AAAANMj8AAAANMj8AAAANMj8AAAANMj8AAAANMj8AAAANMj8AAAANMj8ABoUG644QZNnTq11vf7448/KjIyUhkZGS7t06ZNU2BgoO6+++5y24wZM0a/+93var0WALWPwAMAkl566SXdeuutiouLc2mfPn26fve732nlypUuN1SVpN/85jd66aWXZLfb67FSANVB4AHQ6BUUFOiNN97QhAkTyj1ns9k0YcIEWa1W7dy50+W5+Ph4derUSW+//XZ9lQqgmgg8ABq0oqIiPfroo4qMjFRQUJCuu+46bd261fn8qVOndM899yg0NFTR0dH6/e9/X+602Mcff6zAwEBdffXVbl/j7NmzCgkJ0a5du8o9N3LkSK1atarWjwtA7SLwAGjQnnrqKb3//vt66623lJaWps6dO2vo0KE6ceKEJCkpKUnffPON/vGPf2j9+vX6+uuvlZaW5rKPr7/+Wn369PH4Gr/5zW90+vRpt4Gnf//+2rJli4qKimr3wADUKgIPgAYrPz9fixcv1ty5czV8+HB1795dS5cuVXBwsN544w2dOnVKb731lubNm6ebbrpJ8fHxWrZsmUpLS132c+jQIbVp08bta6Smpur111/XiBEj3AaeNm3aqLi4WNnZ2XVyjABqB4EHgNdNmzZNFoulwse+ffvKbXfw4EGVlJTo2muvdbb5+/urf//+2rt3r/7zn/+opKRE/fv3dz5vs9l02WWXueynsLBQQUFB5fbvcDj00EMPafLkyRo3bpwOHDigkpISlz7BwcGSzs0DAuC7mni7AAB4/PHHdd9991XYp2PHjnX2+uHh4frpp5/Ktb/66qvKzc3V888/r8zMTJWUlGjfvn3q0aOHs0/ZqbOIiIg6qw9AzRF4AHhdREREtQJDp06dFBAQoG+++Ubt27eXJJWUlGjr1q2aOnWqOnbsKH9/f23dulWxsbGSJLvdru+//16DBg1y7ueqq64qd6XVkSNH9Nvf/lYrV65UaGiounTposDAQO3atcsl8OzatUvt2rVTeHh4dQ4dQD3hlBaABis0NFSPPPKInnzySa1bt0579uzRAw88oIKCAk2YMEHNmjVTYmKinnzySW3YsEG7d+92XmJusVic+xk6dKh2797tMsrz6KOPavjw4RoxYoQkqUmTJrr88svLzeP5+uuvdfPNN9fPAQOoNkZ4ADRoc+bMkcPh0NixY3Xq1Cn17dtXn376qVq0aCFJmj9/vh5++GH98pe/VFhYmJ566ikdPnzYZc5Ojx491Lt3b73zzjt66KGH9OGHH+qf//yn9u7d6/JaPXr0cAk8Z86c0QcffKB169bVz8ECqDaLYRiGt4sAgPqSn5+vtm3b6ne/+53LQoMfffSRnnzySe3atUtWa+UGvxcvXqy1a9fqs88+q6tyAdQSRngAmNq2bdu0b98+9e/fX3a7Xc8//7wk6dZbb3XpN2LECB04cEBHjhxRTExMpfbt7++vV199tdZrBlD7GOEBYGrbtm3Tr3/9a+3fv18BAQHq06eP5s+f7zLxGID5EXgAAIDpcZUWAAAwPQIPAAAwPQIPAAAwPQIPAAAwPQIPAAAwPQIPAAAwPQIPAAAwPQIPAAAwPQIPAAAwPQIPAAAwPQIPAAAwvf8PULdALYqVDN8AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "r2vec = np.apply_along_axis(lambda yhat: r2_score(y_test, yhat), axis=1, arr=yhatmat)\n",
    "lam_path = model_gaussian.lambda_\n",
    "plt.plot(-np.log(lam_path), r2vec, linestyle=\"None\", marker=\".\")\n",
    "plt.axhline(r2vec.max(), color=\"red\", linestyle=\"--\", label=f\"max $R^2$ = {r2vec.max():.2f}\")\n",
    "plt.title(r\"Out-of-sample $R^2$ over $-\\log(\\lambda)$\")\n",
    "plt.xlabel(r\"$-\\log(\\lambda$)\")\n",
    "plt.ylabel(r\"$R^2$\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __K-Fold Cross-Validation__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More often, the user wishes to perform model selection via cross-validation before deploying a model.\n",
    "If the model class is initalized with the `\"cv_grpnet\"` solver, \n",
    "we select the best $\\lambda$ by cross-validation and return the model at that $\\lambda$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:00<00:00:00, 21478.51it/s] [dev:51.8%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:00<00:00:00, 31226.25it/s] [dev:50.8%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:00<00:00:00, 39503.28it/s] [dev:57.5%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:00<00:00:00, 39044.08it/s] [dev:52.6%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:00<00:00:00, 33957.77it/s] [dev:53.0%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:00<00:00:00, 43455.42it/s] [dev:52.6%]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-2 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-2 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-2 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-2 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-2 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-2 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GroupElasticNet(solver=&#x27;cv_grpnet&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;GroupElasticNet<span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GroupElasticNet(solver=&#x27;cv_grpnet&#x27;)</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "GroupElasticNet(solver='cv_grpnet')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_cv_gaussian = ad.GroupElasticNet(solver=\"cv_grpnet\")\n",
    "model_cv_gaussian.fit(X_train, y_train, min_ratio=1e-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the `cv_grpnet` backend, calling predict automatically uses the model with the best $\\lambda$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(89,)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_cv_gaussian.predict(X_test).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The class also has a score method that computes $R^2$. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.45673736460682524"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_cv_gaussian.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Binary Classification__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __Logistic Regression__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the `breast_cancer` dataset as a binary classification example and perform a train-test split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((455, 30), (455,), (114, 30), (114,))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = load_breast_cancer()\n",
    "X, y = data.data, data.target\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
    "X_train = np.asfortranarray(X_train)\n",
    "X_test = np.asfortranarray(X_test)\n",
    "X_train.shape, y_train.shape, X_test.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we have a binary classification task, we specify our solver to use the `\"binomial\"` family for our response.\n",
    "We fit using our cross-validation solver as before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 102/102 [00:00:00<00:00:00, 33514.96it/s] [dev:61.6%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:00<00:00:00, 36201.82it/s] [dev:61.8%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:00<00:00:00, 42699.36it/s] [dev:64.6%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:00<00:00:00, 42445.89it/s] [dev:64.7%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:00<00:00:00, 42242.36it/s] [dev:64.2%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:00<00:00:00, 41538.31it/s] [dev:63.3%]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-3 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-3 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-3 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-3 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-3 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-3 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-3 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-3 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-3 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-3 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-3 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GroupElasticNet(family=&#x27;binomial&#x27;, solver=&#x27;cv_grpnet&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;GroupElasticNet<span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GroupElasticNet(family=&#x27;binomial&#x27;, solver=&#x27;cv_grpnet&#x27;)</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "GroupElasticNet(family='binomial', solver='cv_grpnet')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_cv_binomial = ad.GroupElasticNet(solver=\"cv_grpnet\", family=\"binomial\")\n",
    "model_cv_binomial.fit(\n",
    "    X_train, \n",
    "    y_train.astype(np.float64),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the `\"binomial\"` family, the predict method outputs\n",
    "the class label predictions based on the probability estimates.\n",
    "Once we have our class predictions, we can output a contingency table.\n",
    "Here, we see that we never mispredict when the test $y_i = 1$, but have 16 mispredictions when $y_i = 0$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[28, 16],\n",
       "       [ 0, 70]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yhat = model_cv_binomial.predict(X_test)\n",
    "st.contingency.crosstab(y_test, yhat).count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can compute the probability estimates at the test points and plot the frequency."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAHHCAYAAACle7JuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA6XklEQVR4nO3de5yN5f7/8fdizJphDpjBzDBOg0k5VEjjfGywE9ImKsMmHSgMHWZXG6lGKrT31rTrWyNFSpu0S+TsRxQySRiMc2acimFkzOH6/dHD2pY5mFnGrHXPfj0fj/vxcF/3dd/rsy6Hebvu617LZowxAgAAsKBy7i4AAADAVQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZoIyw2WwaPXp0iV1v9uzZstls2rJlyzX7durUSZ06dXLsHzx4UDabTbNnz3a0TZo0STabrcTqAwCJIAPcUJfDwOXNx8dHjRo10ujRo3X8+HF3l+d2r7zyij7//PMSveaaNWucxvzK7f777y/R1wLgfl7uLgD4X/Diiy+qXr16unjxotavX6+EhAQtWbJEO3bsUMWKFd1d3nX75ptvrtnn+eef17PPPuvU9sorr+i+++5T3759S7ymJ598Uq1atXJqq1u3bom/DgD3IsgApaBnz55q2bKlJGnEiBEKCgrS9OnTtXjxYg0aNCjfczIyMlSpUqXSLNNl3t7e1+zj5eUlL6/S+yenffv2uu+++4rUNzs7W7m5uUV6HwA8C7eWADfo0qWLJOnAgQOSpKFDh8rPz08pKSnq1auX/P399cADD0j6I9CMHz9e4eHhstvtioyM1Ouvv66Cvrh+7ty5ioyMlI+Pj1q0aKF169Y5HT906JAef/xxRUZGytfXV0FBQfrzn/+sgwcP5nu9Cxcu6JFHHlFQUJACAgI0ZMgQ/fbbb059rl4jk5+r18jYbDZlZGTogw8+cNz6GTp0qFavXi2bzaZFixbluca8efNks9m0cePGQl+rMJfX77z++uuaOXOmIiIiZLfbtXPnTknS7t27dd9996lq1ary8fFRy5Yt9cUXX+S5zs8//6wuXbrI19dXtWrV0ksvvaT3339fNpvNaSxtNpsmTZqU5/y6detq6NChTm1nzpzR2LFjHb/XDRo00Kuvvqrc3Nx863/nnXcc9bdq1UqbN2/O8zq7d+/WgAEDVK1aNfn6+ioyMlLPPfecJN3wsQZKAzMygBukpKRIkoKCghxt2dnZio6OVrt27fT666+rYsWKMsbonnvu0erVqzV8+HDdeuutWrZsmZ566in98ssvmjFjhtN1165dq08++URPPvmk7Ha73nrrLfXo0UPff/+9mjRpIknavHmzvv32W91///2qVauWDh48qISEBHXq1Ek7d+7Mc6tr9OjRqly5siZNmqTk5GQlJCTo0KFDjrUorvrwww81YsQI3XHHHRo5cqQkKSIiQnfeeafCw8M1d+5c9evXz+mcuXPnKiIiQlFRUde8/rlz53Tq1CmntqpVqzp+nZiYqIsXL2rkyJGy2+2qWrWqfv75Z7Vt21Y1a9bUs88+q0qVKunTTz9V37599e9//9tRT1pamjp37qzs7GxHv3feeUe+vr4uj8eFCxfUsWNH/fLLL3rkkUdUu3Ztffvtt4qLi1Nqaqpmzpzp1H/evHk6d+6cHnnkEdlsNk2bNk333nuv9u/frwoVKkiStm/frvbt26tChQoaOXKk6tatq5SUFP3nP//Ryy+/rE6dOpXIWANuZQDcMImJiUaSWbFihTl58qQ5cuSImT9/vgkKCjK+vr7m6NGjxhhjYmJijCTz7LPPOp3/+eefG0nmpZdecmq/7777jM1mM/v27XO0STKSzJYtWxxthw4dMj4+PqZfv36OtgsXLuSpc+PGjUaSmTNnTp7aW7RoYS5duuRonzZtmpFkFi9e7Gjr2LGj6dixo2P/wIEDRpJJTEx0tE2cONFc/U9OpUqVTExMTJ564uLijN1uN2fOnHG0nThxwnh5eZmJEyfm6X+l1atXO8bi6u3AgQOO2gICAsyJEyeczu3atatp2rSpuXjxoqMtNzfXtGnTxjRs2NDRNnbsWCPJfPfdd071BQYGOl7nMkn51lynTh2n9z5lyhRTqVIls2fPHqd+zz77rClfvrw5fPiwMea/YxsUFGR+/fVXR7/FixcbSeY///mPo61Dhw7G39/fHDp0yOmaubm5jl9fz1gDnoBbS0Ap6Natm6pVq6bw8HDdf//98vPz06JFi1SzZk2nfo899pjT/pIlS1S+fHk9+eSTTu3jx4+XMUZff/21U3tUVJRatGjh2K9du7b69OmjZcuWKScnR5KcZg2ysrJ0+vRpNWjQQJUrV9YPP/yQp/aRI0c6/od/uUYvLy8tWbKkmKNQdEOGDFFmZqY+++wzR9snn3yi7OxsPfjgg0W6xt/+9jctX77caQsJCXEc79+/v6pVq+bY//XXX7Vq1SoNGDDAMZtz6tQpnT59WtHR0dq7d69++eUXSX/8vtx555264447HOdXq1bNcTvQFQsWLFD79u1VpUoVx2ufOnVK3bp1U05OTp5bhAMHDlSVKlUc++3bt5ck7d+/X5J08uRJrVu3Tn/5y19Uu3Ztp3OvnEkribEG3IlbS0ApmDVrlho1aiQvLy/VqFFDkZGRKlfO+f8RXl5eqlWrllPboUOHFBYWJn9/f6f2xo0bO45fqWHDhnleu1GjRrpw4YJOnjypkJAQ/f7774qPj1diYqJ++eUXp7U2Z8+ezXP+1df08/NTaGhogWtqSsJNN92kVq1aae7cuRo+fLikP2513HnnnWrQoEGRrtG0aVN169atwOP16tVz2t+3b5+MMXrhhRf0wgsv5HvOiRMnVLNmTR06dEitW7fOczwyMrJIteVn79692r59u1O4uvq1r3R1OLkcai6vX7ocaC7fUixISYw14E4EGaAU3HHHHY6nlgpit9vzhJsb4YknnlBiYqLGjh2rqKgoBQYGOj5j5cpFpe42ZMgQjRkzRkePHlVmZqY2bdqkf/7znyV2/avXs1x+7xMmTFB0dHS+55TkD/bLM2RXvn737t319NNP59u/UaNGTvvly5fPt58pYBF4YW70WAM3EkEG8GB16tTRihUrdO7cOadZmd27dzuOX2nv3r15rrFnzx5VrFjR8T/9zz77TDExMXrjjTccfS5evKgzZ87kW8PevXvVuXNnx/758+eVmpqqXr16ufy+LitssfD999+v2NhYffzxx/r9999VoUIFDRw48LpfsyD169eXJFWoUKHQmRzpj3HPb6yTk5PztFWpUiXP2F66dEmpqalObRERETp//vw1X7uoLr+fHTt2XLNvaY81UJJYIwN4sF69eiknJyfP/45nzJghm82mnj17OrVv3LjRaZ3LkSNHtHjxYt11112O/8GXL18+z//a//GPf+SZIbjsnXfeUVZWlmM/ISFB2dnZeV7bFZUqVSowQAUHB6tnz5766KOPNHfuXPXo0UPBwcHX/ZoFqV69ujp16qR//etfeUKG9Meak8t69eqlTZs26fvvv3c6Pnfu3DznRURE5Fnf8s477+QZ7wEDBmjjxo1atmxZnmucOXNG2dnZxXo/1apVU4cOHfT+++/r8OHDTseu/v0v7bEGShIzMoAH6927tzp37qznnntOBw8eVPPmzfXNN99o8eLFGjt2rCIiIpz6N2nSRNHR0U6PX0vS5MmTHX3uvvtuffjhhwoMDNTNN9+sjRs3asWKFU6Pgl/p0qVL6tq1qwYMGKDk5GS99dZbateune65557rfn8tWrTQihUrNH36dIWFhalevXpOa0+GDBni+FC7KVOmXPfrXcusWbPUrl07NW3aVA8//LDq16+v48ePa+PGjTp69Kh+/PFHSdLTTz+tDz/8UD169NCYMWMcj1/XqVNH27dvd7rmiBEj9Oijj6p///7q3r27fvzxRy1btixPUHjqqaf0xRdf6O6779bQoUPVokULZWRk6KefftJnn32mgwcPFjtc/P3vf1e7du10++23a+TIkapXr54OHjyor776SklJSU59S3usgRLjzkemgLLu8iPMmzdvLrRfTEyMqVSpUr7Hzp07Z8aNG2fCwsJMhQoVTMOGDc1rr73m9AitMX885jtq1Cjz0UcfmYYNGxq73W5uu+02s3r1aqd+v/32mxk2bJgJDg42fn5+Jjo62uzevTvP48CXa1+7dq0ZOXKkqVKlivHz8zMPPPCAOX36tNM1XX38evfu3aZDhw7G19fXSMrzKHZmZqapUqWKCQwMNL///nuhY3jZ5cevFyxYkO/xy7W99tpr+R5PSUkxQ4YMMSEhIaZChQqmZs2a5u677zafffaZU7/t27ebjh07Gh8fH1OzZk0zZcoU89577+V5/DonJ8c888wzJjg42FSsWNFER0ebffv25RlvY/74vY6LizMNGjQw3t7eJjg42LRp08a8/vrrjkfgC6tf+TzqvWPHDtOvXz9TuXJl4+PjYyIjI80LL7yQ51xXxhrwBDZjXFgZBgClIDs7W2FhYerdu7fee+89d5dzTbNnz9awYcN04MABy32vk9XGGriMNTIAPNbnn3+ukydPasiQIe4upcxjrGFVrJEB4HG+++47bd++XVOmTNFtt92mjh07urukMouxhtUxIwPA4yQkJOixxx5T9erVNWfOHHeXU6Yx1rA61sgAAADLYkYGAABYFkEGAABYVplf7Jubm6tjx47J39+/0I9DBwAAnsMYo3PnziksLKzQ76Er80Hm2LFjCg8Pd3cZAADABUeOHFGtWrUKPF7mg8zlL9o7cuSIAgIC3FwNAAAoivT0dIWHhzt9YW5+ynyQuXw7KSAggCADAIDFXGtZCIt9AQCAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZRFkAACAZXm5uwAAAPCHGcv3uLuEYhvXvZFbX58ZGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFkEGQAAYFluDTIJCQlq1qyZAgICFBAQoKioKH399deO4506dZLNZnPaHn30UTdWDAAAPImXO1+8Vq1amjp1qho2bChjjD744AP16dNH27Zt0y233CJJevjhh/Xiiy86zqlYsaK7ygUAAB7GrUGmd+/eTvsvv/yyEhIStGnTJkeQqVixokJCQtxRHgAA8HAes0YmJydH8+fPV0ZGhqKiohztc+fOVXBwsJo0aaK4uDhduHDBjVUCAABP4tYZGUn66aefFBUVpYsXL8rPz0+LFi3SzTffLEkaPHiw6tSpo7CwMG3fvl3PPPOMkpOTtXDhwgKvl5mZqczMTMd+enr6DX8PAADAPdweZCIjI5WUlKSzZ8/qs88+U0xMjNauXaubb75ZI0eOdPRr2rSpQkND1bVrV6WkpCgiIiLf68XHx2vy5MmlVT4AAHAjt99a8vb2VoMGDdSiRQvFx8erefPmevPNN/Pt27p1a0nSvn37CrxeXFyczp4969iOHDlyQ+oGAADu5/YZmavl5uY63Rq6UlJSkiQpNDS0wPPtdrvsdvuNKA0AAHgYtwaZuLg49ezZU7Vr19a5c+c0b948rVmzRsuWLVNKSormzZunXr16KSgoSNu3b9e4cePUoUMHNWvWzJ1lAwAAD+HWIHPixAkNGTJEqampCgwMVLNmzbRs2TJ1795dR44c0YoVKzRz5kxlZGQoPDxc/fv31/PPP+/OkgEAgAdxa5B57733CjwWHh6utWvXlmI1AADAaty+2BcAAMBVBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZbg0yCQkJatasmQICAhQQEKCoqCh9/fXXjuMXL17UqFGjFBQUJD8/P/Xv31/Hjx93Y8UAAMCTuDXI1KpVS1OnTtXWrVu1ZcsWdenSRX369NHPP/8sSRo3bpz+85//aMGCBVq7dq2OHTume++9150lAwAAD2Izxhh3F3GlqlWr6rXXXtN9992natWqad68ebrvvvskSbt371bjxo21ceNG3XnnnUW6Xnp6ugIDA3X27FkFBATcyNIBALguM5bvcXcJxTaue6Mbct2i/vz2mDUyOTk5mj9/vjIyMhQVFaWtW7cqKytL3bp1c/S56aabVLt2bW3cuLHA62RmZio9Pd1pAwAAZZPbg8xPP/0kPz8/2e12Pfroo1q0aJFuvvlmpaWlydvbW5UrV3bqX6NGDaWlpRV4vfj4eAUGBjq28PDwG/wOAACAu7g9yERGRiopKUnfffedHnvsMcXExGjnzp0uXy8uLk5nz551bEeOHCnBagEAgCfxcncB3t7eatCggSSpRYsW2rx5s958800NHDhQly5d0pkzZ5xmZY4fP66QkJACr2e322W322902QAAwAO4fUbmarm5ucrMzFSLFi1UoUIFrVy50nEsOTlZhw8fVlRUlBsrBAAAnsKtMzJxcXHq2bOnateurXPnzmnevHlas2aNli1bpsDAQA0fPlyxsbGqWrWqAgIC9MQTTygqKqrITywBAICyza1B5sSJExoyZIhSU1MVGBioZs2aadmyZerevbskacaMGSpXrpz69++vzMxMRUdH66233nJnyQAAwIN43OfIlDQ+RwYAYBV8jsx/We5zZAAAAIqLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACyLIAMAACzLy90FAABwI8xYvsfdJaAUMCMDAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsy61BJj4+Xq1atZK/v7+qV6+uvn37Kjk52alPp06dZLPZnLZHH33UTRUDAABP4tYgs3btWo0aNUqbNm3S8uXLlZWVpbvuuksZGRlO/R5++GGlpqY6tmnTprmpYgAA4Enc+jkyS5cuddqfPXu2qlevrq1bt6pDhw6O9ooVKyokJKS0ywMAAB7Oo9bInD17VpJUtWpVp/a5c+cqODhYTZo0UVxcnC5cuFDgNTIzM5Wenu60AQCAssljPtk3NzdXY8eOVdu2bdWkSRNH++DBg1WnTh2FhYVp+/bteuaZZ5ScnKyFCxfme534+HhNnjy5tMoGAABuZDPGGHcXIUmPPfaYvv76a61fv161atUqsN+qVavUtWtX7du3TxEREXmOZ2ZmKjMz07Gfnp6u8PBwnT17VgEBATekdgCA5+ErCkrHuO6Nbsh109PTFRgYeM2f3x4xIzN69Gh9+eWXWrduXaEhRpJat24tSQUGGbvdLrvdfkPqBAAAnsWtQcYYoyeeeEKLFi3SmjVrVK9evWuek5SUJEkKDQ29wdUBAABP59YgM2rUKM2bN0+LFy+Wv7+/0tLSJEmBgYHy9fVVSkqK5s2bp169eikoKEjbt2/XuHHj1KFDBzVr1sydpQMAAA/g1iCTkJAg6Y8PvbtSYmKihg4dKm9vb61YsUIzZ85URkaGwsPD1b9/fz3//PNuqBYAAHgat99aKkx4eLjWrl1bStUAAACr8ajPkQEAACgOggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsl4LM/v37S7oOAACAYnMpyDRo0ECdO3fWRx99pIsXL5Z0TQAAAEXiUpD54Ycf1KxZM8XGxiokJESPPPKIvv/++5KuDQAAoFAuBZlbb71Vb775po4dO6b3339fqampateunZo0aaLp06fr5MmTJV0nAABAHte12NfLy0v33nuvFixYoFdffVX79u3ThAkTFB4eriFDhig1NbWk6gQAAMjjuoLMli1b9Pjjjys0NFTTp0/XhAkTlJKSouXLl+vYsWPq06dPSdUJAACQh5crJ02fPl2JiYlKTk5Wr169NGfOHPXq1Uvlyv2Ri+rVq6fZs2erbt26JVkrAACAE5eCTEJCgv7yl79o6NChCg0NzbdP9erV9d57711XcQAAAIVxKcjs3bv3mn28vb0VExPjyuUBAACKxKU1MomJiVqwYEGe9gULFuiDDz647qIAAACKwqUgEx8fr+Dg4Dzt1atX1yuvvHLdRQEAABSFS0Hm8OHDqlevXp72OnXq6PDhw9ddFAAAQFG4FGSqV6+u7du352n/8ccfFRQUdN1FAQAAFIVLQWbQoEF68skntXr1auXk5CgnJ0erVq3SmDFjdP/995d0jQAAAPly6amlKVOm6ODBg+ratau8vP64RG5uroYMGcIaGQAAUGpcCjLe3t765JNPNGXKFP3444/y9fVV06ZNVadOnZKuDwAAoEAuBZnLGjVqpEaNGpVULQAAAMXiUpDJycnR7NmztXLlSp04cUK5ublOx1etWlUixQEAABTGpSAzZswYzZ49W3/605/UpEkT2Wy2kq4LAADgmlwKMvPnz9enn36qXr16lXQ9AAAARebS49fe3t5q0KBBSdcCAABQLC4FmfHjx+vNN9+UMaak6wEAACgyl24trV+/XqtXr9bXX3+tW265RRUqVHA6vnDhwhIpDgAAoDAuzchUrlxZ/fr1U8eOHRUcHKzAwECnraji4+PVqlUr+fv7q3r16urbt6+Sk5Od+ly8eFGjRo1SUFCQ/Pz81L9/fx0/ftyVsgEAQBnj0oxMYmJiibz42rVrNWrUKLVq1UrZ2dn661//qrvuuks7d+5UpUqVJEnjxo3TV199pQULFigwMFCjR4/Wvffeqw0bNpRIDQAAwLpc/kC87OxsrVmzRikpKRo8eLD8/f117NgxBQQEyM/Pr0jXWLp0qdP+7NmzVb16dW3dulUdOnTQ2bNn9d5772nevHnq0qWLpD9CVOPGjbVp0ybdeeedrpYPAADKAJeCzKFDh9SjRw8dPnxYmZmZ6t69u/z9/fXqq68qMzNTb7/9tkvFnD17VpJUtWpVSdLWrVuVlZWlbt26OfrcdNNNql27tjZu3JhvkMnMzFRmZqZjPz093aVaAACA53NpjcyYMWPUsmVL/fbbb/L19XW09+vXTytXrnSpkNzcXI0dO1Zt27ZVkyZNJElpaWny9vZW5cqVnfrWqFFDaWlp+V4nPj7eab1OeHi4S/UAAADP59KMzP/7f/9P3377rby9vZ3a69atq19++cWlQkaNGqUdO3Zo/fr1Lp1/WVxcnGJjYx376enphBkAAMool4JMbm6ucnJy8rQfPXpU/v7+xb7e6NGj9eWXX2rdunWqVauWoz0kJESXLl3SmTNnnGZljh8/rpCQkHyvZbfbZbfbi10DAACwHpduLd11112aOXOmY99ms+n8+fOaOHFisb62wBij0aNHa9GiRVq1apXq1avndLxFixaqUKGC0+2q5ORkHT58WFFRUa6UDgAAyhCXZmTeeOMNRUdH6+abb9bFixc1ePBg7d27V8HBwfr444+LfJ1Ro0Zp3rx5Wrx4sfz9/R3rXgIDA+Xr66vAwEANHz5csbGxqlq1qgICAvTEE08oKiqKJ5YAAIBrQaZWrVr68ccfNX/+fG3fvl3nz5/X8OHD9cADDzgt/r2WhIQESVKnTp2c2hMTEzV06FBJ0owZM1SuXDn1799fmZmZio6O1ltvveVK2QAAoIyxmTL+hUnp6ekKDAzU2bNnFRAQ4O5yAAClZMbyPe4u4X/CuO6Nbsh1i/rz26UZmTlz5hR6fMiQIa5cFgAAoFhcCjJjxoxx2s/KytKFCxfk7e2tihUrEmQAAECpcOmppd9++81pO3/+vJKTk9WuXbtiLfYFAAC4Hi5/19LVGjZsqKlTp+rBBx/U7t27S+qyHs2q919v1P1MAABKm0szMgXx8vLSsWPHSvKSAAAABXJpRuaLL75w2jfGKDU1Vf/85z/Vtm3bEikMAADgWlwKMn379nXat9lsqlatmrp06aI33nijJOoCAAC4Jpe/awkAAMDdSnSNDAAAQGlyaUYmNja2yH2nT5/uyksAAABck0tBZtu2bdq2bZuysrIUGRkpSdqzZ4/Kly+v22+/3dHPZrOVTJUAAAD5cCnI9O7dW/7+/vrggw9UpUoVSX98SN6wYcPUvn17jR8/vkSLBAAAyI9La2TeeOMNxcfHO0KMJFWpUkUvvfQSTy0BAIBS41KQSU9P18mTJ/O0nzx5UufOnbvuogAAAIrCpSDTr18/DRs2TAsXLtTRo0d19OhR/fvf/9bw4cN17733lnSNAAAA+XJpjczbb7+tCRMmaPDgwcrKyvrjQl5eGj58uF577bUSLRAAAKAgLgWZihUr6q233tJrr72mlJQUSVJERIQqVapUosUBAAAU5ro+EC81NVWpqalq2LChKlWqJGNMSdUFAABwTS4FmdOnT6tr165q1KiRevXqpdTUVEnS8OHDefQaAACUGpeCzLhx41ShQgUdPnxYFStWdLQPHDhQS5cuLbHiAAAACuPSGplvvvlGy5YtU61atZzaGzZsqEOHDpVIYQAAANfi0oxMRkaG00zMZb/++qvsdvt1FwUAAFAULgWZ9u3ba86cOY59m82m3NxcTZs2TZ07dy6x4gAAAArj0q2ladOmqWvXrtqyZYsuXbqkp59+Wj///LN+/fVXbdiwoaRrBAAAyJdLMzJNmjTRnj171K5dO/Xp00cZGRm69957tW3bNkVERJR0jQAAAPkq9oxMVlaWevToobffflvPPffcjagJAACgSIo9I1OhQgVt3779RtQCAABQLC7dWnrwwQf13nvvlXQtAAAAxeLSYt/s7Gy9//77WrFihVq0aJHnO5amT59eIsUBAAAUplhBZv/+/apbt6527Nih22+/XZK0Z88epz42m63kqgMAAChEsYJMw4YNlZqaqtWrV0v64ysJ/v73v6tGjRo3pDgAAIDCFGuNzNXfbv31118rIyOjRAsCAAAoKpcW+152dbABAAAoTcUKMjabLc8aGNbEAAAAdynWGhljjIYOHer4YsiLFy/q0UcfzfPU0sKFC0uuQgAAgAIUK8jExMQ47T/44IMlWgwAAEBxFCvIJCYm3qg6AAAAiu26Fvter3Xr1ql3794KCwuTzWbT559/7nR86NChjnU5l7cePXq4p1gAAOBx3BpkMjIy1Lx5c82aNavAPj169FBqaqpj+/jjj0uxQgAA4Mlc+oqCktKzZ0/17Nmz0D52u10hISGlVBEAALASt87IFMWaNWtUvXp1RUZG6rHHHtPp06cL7Z+Zman09HSnDQAAlE0eHWR69OihOXPmaOXKlXr11Ve1du1a9ezZUzk5OQWeEx8fr8DAQMcWHh5eihUDAIDS5NZbS9dy//33O37dtGlTNWvWTBEREVqzZo26du2a7zlxcXGKjY117KenpxNmAAAoozx6RuZq9evXV3BwsPbt21dgH7vdroCAAKcNAACUTZYKMkePHtXp06cVGhrq7lIAAIAHcOutpfPnzzvNrhw4cEBJSUmqWrWqqlatqsmTJ6t///4KCQlRSkqKnn76aTVo0EDR0dFurBoAAHgKtwaZLVu2qHPnzo79y2tbYmJilJCQoO3bt+uDDz7QmTNnFBYWprvuuktTpkxxfNcTAAD43+bWINOpUycZYwo8vmzZslKsBgAAWI2l1sgAAABciSADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsy61BZt26derdu7fCwsJks9n0+eefOx03xuhvf/ubQkND5evrq27dumnv3r3uKRYAAHgctwaZjIwMNW/eXLNmzcr3+LRp0/T3v/9db7/9tr777jtVqlRJ0dHRunjxYilXCgAAPJGXO1+8Z8+e6tmzZ77HjDGaOXOmnn/+efXp00eSNGfOHNWoUUOff/657r///tIsFQAAeCCPXSNz4MABpaWlqVu3bo62wMBAtW7dWhs3bnRjZQAAwFO4dUamMGlpaZKkGjVqOLXXqFHDcSw/mZmZyszMdOynp6ffmAIBAIDbeWyQcVV8fLwmT57s7jIAoEyZsXyPu0sA8uWxt5ZCQkIkScePH3dqP378uONYfuLi4nT27FnHduTIkRtaJwAAcB+PDTL16tVTSEiIVq5c6WhLT0/Xd999p6ioqALPs9vtCggIcNoAAEDZ5NZbS+fPn9e+ffsc+wcOHFBSUpKqVq2q2rVra+zYsXrppZfUsGFD1atXTy+88ILCwsLUt29f9xUNAAA8hluDzJYtW9S5c2fHfmxsrCQpJiZGs2fP1tNPP62MjAyNHDlSZ86cUbt27bR06VL5+Pi4q2QAAOBB3BpkOnXqJGNMgcdtNptefPFFvfjii6VYFQAAsAqPXSMDAABwLQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWQQZAABgWW799msAuF4zlu9xdwnFNq57I3eXAJQZzMgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADLIsgAAADL8uggM2nSJNlsNqftpptucndZAADAQ3i5u4BrueWWW7RixQrHvpeXx5cMAABKicenAi8vL4WEhLi7DAAA4IE8+taSJO3du1dhYWGqX7++HnjgAR0+fLjQ/pmZmUpPT3faAABA2eTRMzKtW7fW7NmzFRkZqdTUVE2ePFnt27fXjh075O/vn+858fHxmjx5cilXCgBFN2P5HneXAJQZHj0j07NnT/35z39Ws2bNFB0drSVLlujMmTP69NNPCzwnLi5OZ8+edWxHjhwpxYoBAEBp8ugZmatVrlxZjRo10r59+wrsY7fbZbfbS7EqAADgLh49I3O18+fPKyUlRaGhoe4uBQAAeACPDjITJkzQ2rVrdfDgQX377bfq16+fypcvr0GDBrm7NAAA4AE8+tbS0aNHNWjQIJ0+fVrVqlVTu3bttGnTJlWrVs3dpQEAAA/g0UFm/vz57i4BAAB4MI++tQQAAFAYggwAALAsggwAALAsj14jA6B08YmzAKyGGRkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZBBkAAGBZXu4uACiKGcv3uLsEAIAHYkYGAABYFkEGAABYFkEGAABYFkEGAABYFkEGAABYFkEGAABYFkEGAABYFkEGAABYFh+I9z+ID5cDAJQVzMgAAADLIsgAAADLIsgAAADLIsgAAADLskSQmTVrlurWrSsfHx+1bt1a33//vbtLAgAAHsDjg8wnn3yi2NhYTZw4UT/88IOaN2+u6OhonThxwt2lAQAAN/P4IDN9+nQ9/PDDGjZsmG6++Wa9/fbbqlixot5//313lwYAANzMo4PMpUuXtHXrVnXr1s3RVq5cOXXr1k0bN250Y2UAAMATePQH4p06dUo5OTmqUaOGU3uNGjW0e/fufM/JzMxUZmamY//s2bOSpPT09BKv72LG+RK/JgAAVnIjfr5eeV1jTKH9PDrIuCI+Pl6TJ0/O0x4eHu6GagAAKNv+eoOvf+7cOQUGBhZ43KODTHBwsMqXL6/jx487tR8/flwhISH5nhMXF6fY2FjHfm5urn799VcFBQXJZrOVWG3p6ekKDw/XkSNHFBAQUGLXRf4Y79LFeJcuxrv0MNal63rG2xijc+fOKSwsrNB+Hh1kvL291aJFC61cuVJ9+/aV9EcwWblypUaPHp3vOXa7XXa73amtcuXKN6zGgIAA/jKUIsa7dDHepYvxLj2MdelydbwLm4m5zKODjCTFxsYqJiZGLVu21B133KGZM2cqIyNDw4YNc3dpAADAzTw+yAwcOFAnT57U3/72N6WlpenWW2/V0qVL8ywABgAA/3s8PshI0ujRowu8leQudrtdEydOzHMbCzcG4126GO/SxXiXHsa6dJXGeNvMtZ5rAgAA8FAe/YF4AAAAhSHIAAAAyyLIAAAAyyLIAAAAyyLIFGLWrFmqW7eufHx81Lp1a33//feF9l+wYIFuuukm+fj4qGnTplqyZEkpVVo2FGe83333XbVv315VqlRRlSpV1K1bt2v+/sBZcf98XzZ//nzZbDbHh1Ti2oo71mfOnNGoUaMUGhoqu92uRo0a8e9JMRR3vGfOnKnIyEj5+voqPDxc48aN08WLF0upWmtbt26devfurbCwMNlsNn3++efXPGfNmjW6/fbbZbfb1aBBA82ePfv6ijDI1/z58423t7d5//33zc8//2wefvhhU7lyZXP8+PF8+2/YsMGUL1/eTJs2zezcudM8//zzpkKFCuann34q5cqtqbjjPXjwYDNr1iyzbds2s2vXLjN06FATGBhojh49WsqVW1Nxx/uyAwcOmJo1a5r27dubPn36lE6xFlfcsc7MzDQtW7Y0vXr1MuvXrzcHDhwwa9asMUlJSaVcuTUVd7znzp1r7Ha7mTt3rjlw4IBZtmyZCQ0NNePGjSvlyq1pyZIl5rnnnjMLFy40ksyiRYsK7b9//35TsWJFExsba3bu3Gn+8Y9/mPLly5ulS5e6XANBpgB33HGHGTVqlGM/JyfHhIWFmfj4+Hz7DxgwwPzpT39yamvdurV55JFHbmidZUVxx/tq2dnZxt/f33zwwQc3qsQyxZXxzs7ONm3atDH/93//Z2JiYggyRVTcsU5ISDD169c3ly5dKq0Sy5TijveoUaNMly5dnNpiY2NN27Ztb2idZVFRgszTTz9tbrnlFqe2gQMHmujoaJdfl1tL+bh06ZK2bt2qbt26OdrKlSunbt26aePGjfmes3HjRqf+khQdHV1gf/yXK+N9tQsXLigrK0tVq1a9UWWWGa6O94svvqjq1atr+PDhpVFmmeDKWH/xxReKiorSqFGjVKNGDTVp0kSvvPKKcnJySqtsy3JlvNu0aaOtW7c6bj/t379fS5YsUa9evUql5v81N+JnpSU+2be0nTp1Sjk5OXm+BqFGjRravXt3vuekpaXl2z8tLe2G1VlWuDLeV3vmmWcUFhaW5y8I8nJlvNevX6/33ntPSUlJpVBh2eHKWO/fv1+rVq3SAw88oCVLlmjfvn16/PHHlZWVpYkTJ5ZG2ZblyngPHjxYp06dUrt27WSMUXZ2th599FH99a9/LY2S/+cU9LMyPT1dv//+u3x9fYt9TWZkYHlTp07V/PnztWjRIvn4+Li7nDLn3Llzeuihh/Tuu+8qODjY3eWUebm5uapevbreeecdtWjRQgMHDtRzzz2nt99+292llUlr1qzRK6+8orfeeks//PCDFi5cqK+++kpTpkxxd2koImZk8hEcHKzy5cvr+PHjTu3Hjx9XSEhIvueEhIQUqz/+y5Xxvuz111/X1KlTtWLFCjVr1uxGlllmFHe8U1JSdPDgQfXu3dvRlpubK0ny8vJScnKyIiIibmzRFuXKn+3Q0FBVqFBB5cuXd7Q1btxYaWlpunTpkry9vW9ozVbmyni/8MILeuihhzRixAhJUtOmTZWRkaGRI0fqueeeU7ly/H+/JBX0szIgIMCl2RiJGZl8eXt7q0WLFlq5cqWjLTc3VytXrlRUVFS+50RFRTn1l6Tly5cX2B//5cp4S9K0adM0ZcoULV26VC1btiyNUsuE4o73TTfdpJ9++klJSUmO7Z577lHnzp2VlJSk8PDw0izfUlz5s922bVvt27fPERYlac+ePQoNDSXEXIMr433hwoU8YeVyiDR8FWGJuyE/K11eJlzGzZ8/39jtdjN79myzc+dOM3LkSFO5cmWTlpZmjDHmoYceMs8++6yj/4YNG4yXl5d5/fXXza5du8zEiRN5/LoYijveU6dONd7e3uazzz4zqampju3cuXPueguWUtzxvhpPLRVdccf68OHDxt/f34wePdokJyebL7/80lSvXt289NJL7noLllLc8Z44caLx9/c3H3/8sdm/f7/55ptvTEREhBkwYIC73oKlnDt3zmzbts1s27bNSDLTp08327ZtM4cOHTLGGPPss8+ahx56yNH/8uPXTz31lNm1a5eZNWsWj1/fSP/4xz9M7dq1jbe3t7njjjvMpk2bHMc6duxoYmJinPp/+umnplGjRsbb29vccsst5quvvirliq2tOONdp04dIynPNnHixNIv3KKK++f7SgSZ4inuWH/77bemdevWxm63m/r165uXX37ZZGdnl3LV1lWc8c7KyjKTJk0yERERxsfHx4SHh5vHH3/c/Pbbb6VfuAWtXr0633+LL49xTEyM6dixY55zbr31VuPt7W3q169vEhMTr6sGmzHMnQEAAGtijQwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggwAALAsggzwP2zo0KHq27fvdV3j4MGDstlshX4z9po1a2Sz2XTmzBlJ0uzZs1W5cmXH8UmTJunWW2+9rjpclZaWpu7du6tSpUpONV2puONUlDEpik6dOmns2LHXdQ2grCPIABYwdOhQ2Ww22Ww2eXt7q0GDBnrxxReVnZ3t7tKKpE2bNkpNTVVgYGC+xydMmOD0/SslEbCKasaMGUpNTVVSUpL27NlTKq9ZUlJTUzV48GA1atRI5cqVI/TgfxJBBrCIHj16KDU1VXv37tX48eM1adIkvfbaa/n2vXTpUilXVzhvb2+FhITIZrPle9zPz09BQUGlXNUfUlJS1KJFCzVs2FDVq1d3Sw2uyszMVLVq1fT888+refPm7i4HcAuCDGARdrtdISEhqlOnjh577DF169ZNX3zxhaT/zmC8/PLLCgsLU2RkpCTpp59+UpcuXeTr66ugoCCNHDlS58+fz3PtyZMnq1q1agoICNCjjz7qFISWLl2qdu3aqXLlygoKCtLdd9+tlJSUPNfYvXu32rRpIx8fHzVp0kRr1651HLv61tLVrry1NGnSJH3wwQdavHixYxZqzZo16tKli0aPHu103smTJ+Xt7Z3n23SvlJCQoIiICHl7eysyMlIffvih41jdunX173//W3PmzJHNZtPQoUMLvM6VSmJMJGnHjh3q2bOn/Pz8VKNGDT300EM6depUkWq4XP+bb76pIUOGFDjbBZR1BBnAonx9fZ0Cx8qVK5WcnKzly5fryy+/VEZGhqKjo1WlShVt3rxZCxYs0IoVK/KEgZUrV2rXrl1as2aNPv74Yy1cuFCTJ092HM/IyFBsbKy2bNmilStXqly5curXr59yc3OdrvPUU09p/Pjx2rZtm6KiotS7d2+dPn262O9rwoQJGjBggGMGKjU1VW3atNGIESM0b948ZWZmOvp+9NFHqlmzprp06ZLvtRYtWqQxY8Zo/Pjx2rFjhx555BENGzZMq1evliRt3rxZPXr00IABA5Samqo333yzSDWWxJicOXNGXbp00W233aYtW7Zo6dKlOn78uAYMGFDsMQP+p13XV04CKBVXftt0bm6uWb58ubHb7WbChAmO4zVq1DCZmZmOc9555x1TpUoVc/78eUfbV199ZcqVK2fS0tIc51WtWtVkZGQ4+iQkJBg/Pz+Tk5OTby0nT540ksxPP/1kjDHmwIEDRpKZOnWqo09WVpapVauWefXVV40x//2G3MvfKJyYmGgCAwMd/SdOnGiaN2+e7/u97PfffzdVqlQxn3zyiaOtWbNmZtKkSQWOW5s2bczDDz/s1PbnP//Z9OrVy7Hfp0+fQr/pu6B6ruTKmEyZMsXcddddTtc5cuSIkWSSk5ONMX98U/OYMWMKre2y4vQFyhJmZACL+PLLL+Xn5ycfHx/17NlTAwcO1KRJkxzHmzZtKm9vb8f+rl271Lx5c1WqVMnR1rZtW+Xm5io5OdnR1rx5c1WsWNGxHxUVpfPnz+vIkSOSpL1792rQoEGqX7++AgICVLduXUnS4cOHneqLiopy/NrLy0stW7bUrl27SuS9S5KPj48eeughvf/++5KkH374QTt27Cj0dtCuXbvUtm1bp7a2bdted10lMSY//vijVq9eLT8/P8d20003SVK+t6kA5M/L3QUAKJrOnTsrISFB3t7eCgsLk5eX81/fKwNLSerdu7fq1Kmjd999V2FhYcrNzVWTJk3csqB4xIgRuvXWW3X06FElJiaqS5cuqlOnTqnXURJjcv78efXu3VuvvvpqnmOhoaElWS5QpjEjA1hEpUqV1KBBA9WuXTtPiMlP48aN9eOPPyojI8PRtmHDBpUrV86xGFj6Y2bg999/d+xv2rRJfn5+Cg8P1+nTp5WcnKznn39eXbt2VePGjfXbb7/l+3qbNm1y/Do7O1tbt25V48aNXXmr8vb2Vk5OTp72pk2bqmXLlnr33Xc1b948/eUvfyn0Oo0bN9aGDRuc2jZs2KCbb77ZpbokldiY3H777fr5559Vt25dNWjQwGm7UaEUKIsIMkAZ9cADD8jHx0cxMTHasWOHVq9erSeeeEIPPfSQatSo4eh36dIlDR8+XDt37tSSJUs0ceJEjR49WuXKlVOVKlUUFBSkd955R/v27dOqVasUGxub7+vNmjVLixYt0u7duzVq1Cj99ttv1wwaBalbt662b9+u5ORknTp1SllZWY5jI0aM0NSpU2WMUb9+/Qq9zlNPPaXZs2crISFBe/fu1fTp07Vw4UJNmDDBpbokldiYjBo1Sr/++qsGDRqkzZs3KyUlRcuWLdOwYcPyDXEFSUpKUlJSks6fP6+TJ08qKSlJO3fudPn9AVZDkAHKqIoVK2rZsmX69ddf1apVK913333q2rWr/vnPfzr169q1qxo2bKgOHTpo4MCBuueeexxrb8qVK6f58+dr69atatKkicaNG1fgZ9dMnTpVU6dOVfPmzbV+/Xp98cUXCg4Odqn2hx9+WJGRkWrZsqWqVavmNKsyaNAgeXl5adCgQfLx8Sn0On379tWbb76p119/Xbfccov+9a9/KTExUZ06dXKpLqnkxiQsLEwbNmxQTk6O7rrrLjVt2lRjx45V5cqVVa5c0f9pvu2223Tbbbdp69atmjdvnm677Tb16tXL5fcHWI3NGGPcXQQAFNXBgwcVERGhzZs36/bbb3d3OQDcjCADwBKysrJ0+vRpTZgwQQcOHMiz9gXA/yZuLQGwhA0bNig0NFSbN2/W22+/7e5yAHgIZmQAAIBlMSMDAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAsiyADAAAs6/8DOQY13srDqawAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(model_cv_binomial.predict_proba(X_test)[:, 1], alpha = 0.5)\n",
    "plt.xlabel(\"Probability of label 1\")\n",
    "plt.ylabel(\"Frequency\")\n",
    "plt.title(\"Probability Frequency\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Multi-Class Classification__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### __Multinomial Regression__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the `digits` dataset as a multi-class classification example and perform a train-test split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1437, 64), (1437,), (1437, 10), (360, 64), (360,))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = load_digits()\n",
    "X, y = data.data, data.target\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
    "X_train = np.asfortranarray(X_train)\n",
    "X_test = np.asfortranarray(X_test)\n",
    "# one-hot-encode labels\n",
    "oh = OneHotEncoder(sparse_output=False)\n",
    "y_train2 = oh.fit_transform(y_train[:, np.newaxis])\n",
    "X_train.shape, y_train.shape, y_train2.shape, X_test.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we have a multi-class classification task, \n",
    "we specify our solver to use the `\"multinomial\"` family for our response.\n",
    "We fit using our cross-validation solver as before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:01<00:00:00, 75.73it/s] [dev:84.6%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:01<00:00:00, 79.43it/s] [dev:84.9%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:01<00:00:00, 72.49it/s] [dev:84.8%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:01<00:00:00, 75.69it/s] [dev:85.0%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 101/101 [00:00:01<00:00:00, 77.03it/s] [dev:85.1%]\n",
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 100/100 [00:00:01<00:00:00, 75.64it/s] [dev:84.7%]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-6 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-6 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-6 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-6 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-6 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-6 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-6 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-6 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-6 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-6 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-6 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-6 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-6 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-6\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GroupElasticNet(family=&#x27;multinomial&#x27;, solver=&#x27;cv_grpnet&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" checked><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;GroupElasticNet<span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GroupElasticNet(family=&#x27;multinomial&#x27;, solver=&#x27;cv_grpnet&#x27;)</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "GroupElasticNet(family='multinomial', solver='cv_grpnet')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_cv_multinomial = ad.GroupElasticNet(solver=\"cv_grpnet\", family=\"multinomial\")\n",
    "model_cv_multinomial.fit(\n",
    "    X_train, \n",
    "    y_train2.astype(np.float64),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the `\"multinomial\"` family, the predict method outputs\n",
    "the class label predictions based on the probability estimates just like the `\"binomial\"` family.\n",
    "Once we have our class predictions, we can output a contingency table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[29,  0,  0,  0,  0,  0,  0,  0,  0,  0],\n",
       "       [ 0, 32,  0,  0,  0,  0,  0,  0,  2,  2],\n",
       "       [ 0,  0, 41,  1,  0,  0,  0,  0,  0,  1],\n",
       "       [ 0,  1,  0, 34,  0,  0,  0,  0,  1,  0],\n",
       "       [ 0,  1,  0,  0, 37,  0,  0,  1,  0,  1],\n",
       "       [ 0,  0,  0,  0,  0, 37,  0,  0,  0,  3],\n",
       "       [ 0,  1,  0,  0,  0,  0, 37,  0,  0,  0],\n",
       "       [ 0,  0,  0,  0,  0,  1,  0, 31,  0,  0],\n",
       "       [ 0,  3,  0,  1,  0,  2,  0,  2, 23,  0],\n",
       "       [ 0,  0,  0,  0,  0,  0,  0,  1,  0, 34]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yhat = model_cv_multinomial.predict(X_test)\n",
    "st.contingency.crosstab(y_test, yhat).count"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "adelie",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}