{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# __Frequently-Asked Questions__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import adelie as ad\n",
    "import logging\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __How to properly solve for a single regularization value?__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The bread-and-butter of our solver is block-coordinate descent,\n",
    "which works stunningly well if it is warm-started with a good initial value (like most algorithms).\n",
    "As such, we strongly recommend users to _always_ solve the group elastic net \n",
    "using a path of regularization values.\n",
    "`adelie` auto-generates an evenly-spaced path on the log-scale \n",
    "(if the path is not provided) to start from $\\lambda_{\\max}$,\n",
    "the smallest $\\lambda$ such that the optimal penalized coefficients are exactly $0$,\n",
    "to `min_ratio` times $\\lambda_{\\max}$.\n",
    "If the user is only interested in the solution at a particular $\\lambda^\\star$,\n",
    "we recommend the methods discussed below.\n",
    "\n",
    "For simplicity, we work under the lasso setting \n",
    "though the discussion carries to the general group elastic net setting as well.\n",
    "We first generate a dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 100             # number of samples\n",
    "p = 200             # number of features\n",
    "seed = 0            # random seed\n",
    "lmda_star = 1e-2    # user-specified lambda\n",
    "\n",
    "np.random.seed(seed)\n",
    "X = np.asfortranarray(np.random.normal(0, 1, (n, p)))\n",
    "y = X[:,0] * np.random.normal(0, 1) + np.random.normal(0, 1, n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we run a \"dry-run\" of the solver to find $\\lambda_{\\max}$.\n",
    "We recommend this method in general since $\\lambda_{\\max}$ is difficult to determine\n",
    "when some coefficients are unpenalized (e.g. $0$ penalty factor for some groups)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "with ad.logger.logger_level(logging.ERROR):\n",
    "    state = ad.grpnet(X, ad.glm.gaussian(y), lmda_path_size=0, progress_bar=False)\n",
    "    lmda_max = state.lmda_max"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We discuss the first method to solve for $\\lambda^\\star$.\n",
    "The idea is to generate an evenly-spaced path on the log-scale from $\\lambda_{\\max}$ to $\\lambda^\\star$.\n",
    "The easiest way to do this is to set `min_ratio` such that `min_ratio` times $\\lambda_{\\max}$ is precisely $\\lambda^\\star$.\n",
    "Note that if `min_ratio` is larger than $1$, then the solution at `min_ratio` times $\\lambda_{\\max}$\n",
    "is equivalent to that at $\\lambda_{\\max}$.\n",
    "Moreover, the user can provide the fineness of the gridding via `lmda_path_size` argument.\n",
    "Finally, we set `early_exit` to `False` so that we always fit until the end of the path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 20/20 [00:00:00<00:00:00, 4873.59it/s] [dev:98.0%]\n"
     ]
    }
   ],
   "source": [
    "min_ratio = lmda_star / lmda_max    # min_ratio * lmda_max == lmda_star\n",
    "lmda_path_size = 20                 # number of grid points on the path\n",
    "state = ad.grpnet(\n",
    "    X, \n",
    "    ad.glm.gaussian(y), \n",
    "    min_ratio=min(1, min_ratio), \n",
    "    lmda_path_size=lmda_path_size,\n",
    "    early_exit=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now verify that the last fitted regularization is indeed $\\lambda^\\star$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert state.lmdas[-1] == lmda_star"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The more general method is to provide a path of $\\lambda$ directly to the solver.\n",
    "Suppose the user wishes to provide a differently generated path from $\\lambda_{\\max}$ to $\\lambda^\\star$\n",
    "(e.g. evenly-spaced on the original space).\n",
    "Then, we may run the solver with the following arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m\u001b[1;32m█\u001b[0m| 20/20 [00:00:00<00:00:00, 4388.17it/s] [dev:98.0%]\n"
     ]
    }
   ],
   "source": [
    "lmda_path = np.linspace(lmda_max, lmda_star, num=lmda_path_size, endpoint=True)\n",
    "state = ad.grpnet(\n",
    "    X,\n",
    "    ad.glm.gaussian(y),\n",
    "    lmda_path=lmda_path,\n",
    "    early_exit=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, we verify that the last fitted regularization is indeed $\\lambda^\\star$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert state.lmdas[-1] == lmda_star"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "adelie",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}