EvoLinear.jl

Appearance

API ​

Regressors ​

EvoLinear.EvoLinearRegressor Type
julia
EvoLinearRegressor(; kwargs...)

A model type for constructing a EvoLinearRegressor, based on EvoLinear.jl, and implementing both an internal API and the MLJ model interface.

Keyword arguments

  • loss=:mse: loss function to be minimised. Can be one of:

    • :mse

    • :logloss

    • :poisson

    • :gamma

    • :tweedie

  • metric: The evaluation metric used to track evaluation data and serves as a basis for early stopping. Supported metrics are:

    • :mse: Mean squared error. Adapted for general regression models.

    • :rmse: Root mean squared error. Adapted for general regression models.

    • :mae: Mean absolute error. Adapted for general regression models.

    • :logloss: Adapted for logistic regression models.

    • :poisson: Poisson deviance. Adapted for count models.

    • :gamma: Gamma deviance. Adapted to regression problem on Gamma like, positively distributed targets.

    • :tweedie: Tweedie deviance. Adapted to regression problem on Tweedie like, positively distributed targets with probability mass at y == 0.

  • nrounds=10: maximum number of training rounds.

  • eta=1: Learning rate. Typically in the range [1e-2, 1].

  • L1=0: Regularization penalty applied by shrinking to 0 weight update if update is < L1. No penalty if update > L1. Results in sparse feature selection. Typically in the [0, 1] range on normalized features.

  • L2=0: Regularization penalty applied to the squared of the weight update value. Restricts large parameter values. Typically in the [0, 1] range on normalized features.

  • seed::Int=123: random seed.

  • updater=:all: training method. Only :all is supported at the moment. Gradients for each feature are computed simultaneously, then bias is updated based on all features update.

Internal API

Use config = EvoLinearRegressor() to construct an hyper-parameter struct with default hyper-parameters. Provide keyword arguments as listed above to override defaults, for example:

julia
EvoLinearRegressor(loss=:logloss, L1=1e-3, L2=1e-2, nrounds=100)

Training model

A model is built using fit:

julia
config = EvoLinearRegressor()
m = fit(config, date; feature_names, target_name)

Inference

Fitted results is an EvoLinearModel which acts as a prediction function when passed a features matrix as argument.

julia
preds = m(x)

MLJ Interface

From MLJ, the type can be imported using:

julia
EvoLinearRegressor = @load EvoLinearRegressor pkg=EvoLinear

Do model = EvoLinearRegressor() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in EvoLinearRegressor(loss=...).

Training model

In MLJ or MLJBase, bind an instance model to data with mach = machine(model, X, y) where:

  • X: any table of input features (eg, a DataFrame) whose columns each have one of the following element scitypes: Continuous, Count, or <:OrderedFactor; check column scitypes with schema(X)

  • y: is the target, which can be any AbstractVector whose element scitype is <:Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Operations

  • predict(mach, Xnew): return predictions of the target given

features Xnew having the same scitype as X above. Predictions are deterministic.

Fitted parameters

The fields of fitted_params(mach) are:

  • :fitresult: the EvoLinearModel object returned by EvoLnear.jl fitting algorithm.

Report

The fields of report(mach) are:

  • :coef: Vector of coefficients (βs) associated to each of the features.

  • :bias: Value of the bias.

  • :names: Names of each of the features.

source

Training ​

MLJModelInterface.fit Function
julia
function fit(
    learner::EvoLinearRegressor,
    dtrain;
    feature_names,
    target_name,
    weight_name=nothing,
    deval=nothing,
    metric=nothing,
    print_every_n=9999,
    early_stopping_rounds=9999,
    verbosity=1
)

Provided a config, EvoLinear.fit takes x and y as features and target inputs, plus optionally w as weights and train a Linear boosted model.

Arguments

  • learner::EvoLinearRegressor:

  • dtrain: A Tables.jl compatible table containing the feature, target and optionally weight variables.

Keyword arguments

  • `target_name:

  • `feature_names=nothing:

  • `weight_name=nothing:

  • `deval=nothing:

  • `print_every_n=9999:

  • `verbosity=1:

source

Inference ​

MLJModelInterface.predict Function
julia
predict(m::EvoLinearModel, data; proj::Bool=true)

Predictions from an EvoLinear model.

source

Metrics ​

EvoLinear.Metrics.gamma_deviance Method
julia
gamma_deviance(p, y)
gamma_deviance(p, y, w)

Gamma deviance evaluation metric. 𝐷 = 2 * (log(μ/y) + y/μ - 1)

Arguments

  • p: predicted value. Assumes that p is on a projected basis (ie. in the [0-Inf] range).

  • y: observed target variable.

  • w: vector of weights.

source
EvoLinear.Metrics.logloss Method
julia
logloss(p, y)
logloss(p, y, w)

Logloss evaluation metric. ylog(p) + (1-y)log(1-p)

Arguments

  • p: predicted value. Assumes that p is on a projected basis (ie. in the [0-1] range).

  • y: observed target variable.

  • w: vector of weights.

source
EvoLinear.Metrics.mae Method
julia
mae(p, y)
mae(p, y, w)

Mean absolute error evaluation metric.

Arguments

  • p: predicted value.

  • y: observed target variable.

  • w: vector of weights.

source
EvoLinear.Metrics.mse Method
julia
mse(p, y)
mse(p, y, w)

Mean squared error evaluation metric.

Arguments

  • p: predicted value.

  • y: observed target variable.

  • w: vector of weights.

source
EvoLinear.Metrics.poisson_deviance Method
julia
poisson_deviance(p, y)
poisson_deviance(p, y, w)

Poisson deviance evaluation metric. 𝐷 = 2 * (y * log(y/p) + p - y)

Arguments

  • p: predicted value. Assumes that p is on a projected basis (ie. in the [0-Inf] range).

  • y: observed target variable.

  • w: vector of weights.

source
EvoLinear.Metrics.rmse Method
julia
rmse(p, y)
rmse(p, y, w)

Root-Mean squared error evaluation metric.

Arguments

  • p: predicted value.

  • y: observed target variable.

  • w: vector of weights

source
EvoLinear.Metrics.tweedie_deviance Method
julia
tweedie_deviance(p, y)
tweedie_deviance(p, y, w)

Tweedie deviance evaluation metric. Fixed rho (ρ) of 1.5. 𝐷 = 2 * (y²⁻ʳʰᵒ/(1-rho)(2-rho) - yμ¹⁻ʳʰᵒ/(1-rho) + μ²⁻ʳʰᵒ/(2-rho))

Arguments

  • p: predicted value. Assumes that p is on a projected basis (ie. in the [0-Inf] range).

  • y: observed target variable.

  • w: vector of weights.

source