adelie.solver.gaussian_cov#

adelie.solver.gaussian_cov(A: ndarray | MatrixCovBase32 | MatrixCovBase64, v: ndarray, *, constraints: list[ConstraintBase32 | ConstraintBase64] | None = None, groups: ndarray | None = None, alpha: float = 1, penalty: ndarray | None = None, lmda_path: ndarray | None = None, max_iters: int = 100000, tol: float = 1e-07, rdev_tol: float = 0.001, newton_tol: float = 1e-12, newton_max_iters: int = 1000, n_threads: int = 1, early_exit: bool = True, screen_rule: str = 'pivot', min_ratio: float = 0.01, lmda_path_size: int = 100, max_screen_size: int | None = None, max_active_size: int | None = None, pivot_subset_ratio: float = 0.1, pivot_subset_min: int = 1, pivot_slack_ratio: float = 1.25, check_state: bool = False, progress_bar: bool = True, warm_start=None, exit_cond: Callable | None = None)[source]#

Solves Gaussian group elastic net via covariance method.

The Gaussian elastic net problem via covariance method minimizes the following:

\[\begin{align*} \mathrm{minimize}_{\beta} \quad& \frac{1}{2} \beta^\top A \beta - v^\top \beta + \lambda \sum\limits_{g=1}^G \omega_g \left( \alpha \|\beta_g\|_2 + \frac{1-\alpha}{2} \|\beta_g\|_2^2 \right) \end{align*}\]

where \(\beta\) is the coefficient vector, \(A\) is any positive semi-definite matrix, \(v\) is any vector, \(\lambda \geq 0\) is the regularization parameter, \(G\) is the number of groups, \(\omega \geq 0\) is the penalty factor, \(\alpha \in [0,1]\) is the elastic net parameter, and \(\beta_g\) are the coefficients for the \(g\) th group.

Parameters:

A(p, p) Union[ndarray, MatrixCovBase64, MatrixCovBase32]

Positive semi-definite matrix. It is typically one of the matrices defined in adelie.matrix submodule or numpy.ndarray.

v(p,) ndarray

Linear term.

constraints(G,) list[Union[ConstraintBase32, ConstraintBase64]], optional

List of constraints for each group. constraints[i] is the constraint object corresponding to group i. If constraints[i] is None, then the i th group is unconstrained. If None, every group is unconstrained. Default is None.

groups(G,) ndarray, optional

List of starting indices to each group where G is the number of groups. groups[i] is the starting index of the i th group. Default is None, in which case it is set to np.arange(p).

alphafloat, optional

Elastic net parameter. It must be in the range \([0,1]\). Default is 1.

penalty(G,) ndarray, optional

Penalty factor for each group in the same order as groups. It must be a non-negative vector. Default is None, in which case, it is set to np.sqrt(group_sizes).

lmda_path(L,) ndarray, optional

The regularization path to solve for. The full path is not considered if early_exit is True. It is recommended that the path is sorted in decreasing order. If None, the path will be generated. Default is None.

max_itersint, optional

Maximum number of coordinate descents. Default is int(1e5).

tolfloat, optional

Coordinate descent convergence tolerance. Default is 1e-7.

rdev_tolfloat, optional

Relative percent deviance explained tolerance. Default is 1e-3.

newton_tolfloat, optional

Convergence tolerance for the BCD update. Default is 1e-12.

newton_max_itersint, optional

Maximum number of iterations for the BCD update. Default is 1000.

n_threadsint, optional

Number of threads. Default is 1.

early_exitbool, optional

True if the function should early exit based on training deviance explained. Default is True.

min_ratiofloat, optional

The ratio between the largest and smallest \(\lambda\) in the regularization sequence if it is to be generated. Default is 1e-2.

lmda_path_sizeint, optional

Number of regularizations in the path if it is to be generated. Default is 100.

screen_rulestr, optional

The type of screening rule to use. It must be one of the following options:

"strong": adds groups whose active scores are above the strong threshold.

"pivot": adds groups whose active scores are above the pivot cutoff with slack.

Default is "pivot".

max_screen_sizeint, optional

Maximum number of screen groups allowed. The function will return a valid state and guarantees to have screen set size less than or equal to max_screen_size. If None, it will be set to the total number of groups. Default is None.

max_active_sizeint, optional

Maximum number of active groups allowed. The function will return a valid state and guarantees to have active set size less than or equal to max_active_size. If None, it will be set to the total number of groups. Default is None.

pivot_subset_ratiofloat, optional

If screening takes place, then the (1 + pivot_subset_ratio) * s largest active scores are used to determine the pivot point where s is the current screen set size. It is only used if screen_rule="pivot". Default is 0.1.

pivot_subset_minint, optional

If screening takes place, then at least pivot_subset_min number of active scores are used to determine the pivot point. It is only used if screen_rule="pivot". Default is 1.

pivot_slack_ratiofloat, optional

If screening takes place, then pivot_slack_ratio number of groups with next smallest (new) active scores below the pivot point are also added to the screen set as slack. It is only used if screen_rule="pivot". Default is 1.25.

check_statebool, optional

True is state should be checked for inconsistencies before calling solver. Default is False.

Warning

The check may take a long time if the inputs are big!

progress_barbool, optional

True to enable progress bar. Default is True.

warm_startoptional

If no warm-start is provided, the initial solution is set to 0 and other invariance quantities are set accordingly. Otherwise, the warm-start is used to extract all necessary state variables. If warm-start is used, the user must still provide consistent inputs, that is, warm-start will not overwrite most arguments passed into this function. However, changing configuration settings such as tolerance levels is well-defined. Default is None.

Note

The primary use-case is when a user already called the function with warm_start=False but would like to continue fitting down a longer path of regularizations. This way, the user does not have to restart the fit at the beginning, but can simply continue from the last returned state.

Warning

We have only tested warm-starts in the setting described in the note above, that is, when lmda_path and possibly static configurations have changed. Use with caution in other settings!

exit_condCallable, optional

If not None, it must be a callable object that takes in a single argument. The argument is the current state object of the same type as the return value. During the optimization, after obtaining the solution at each regularization value, exit_cond(state) is evaluated as an opportunity for the user to early exit the program based on their own rule. Default is None.

Note

The algorithm early exits if exit_cond(state) evaluates to True or the built-in early exit function evaluates to True (if early_exit is True).

Returns:

state: The resulting state after running the solver.