# Abstraction for Statistical Models

This package defines an abstract type StatisticalModel, and an abstract subtype RegressionModel.

Particularly, instances of StatisticalModel implement the following methods.

StatsBase.adjr2Function
adjr2(model::StatisticalModel)
adjr²(model::StatisticalModel)

For linear models, the adjusted R² is defined as $1 - (1 - (1-R^2)(n-1)/(n-p))$, with $R^2$ the coefficient of determination, $n$ the number of observations, and $p$ the number of coefficients (including the intercept). This definition is generally known as the Wherry Formula I.

source
adjr2(model::StatisticalModel, variant::Symbol)
adjr²(model::StatisticalModel, variant::Symbol)

For nonlinear models, one of the several pseudo R² definitions must be chosen via variant. The only currently supported variants are :MacFadden, defined as $1 - (\log (L) - k)/\log (L0)$ and :devianceratio, defined as $1 - (D/(n-k))/(D_0/(n-1))$. In these formulas, $L$ is the likelihood of the model, $L0$ that of the null model (the model including only the intercept), $D$ is the deviance of the model, $D_0$ is the deviance of the null model, $n$ is the number of observations (given by nobs) and $k$ is the number of consumed degrees of freedom of the model (as returned by dof).

source
StatsBase.aicFunction
aic(model::StatisticalModel)

Akaike's Information Criterion, defined as $-2 \log L + 2k$, with $L$ the likelihood of the model, and k its number of consumed degrees of freedom (as returned by dof).

source
StatsBase.aiccFunction
aicc(model::StatisticalModel)

Corrected Akaike's Information Criterion for small sample sizes (Hurvich and Tsai 1989), defined as $-2 \log L + 2k + 2k(k-1)/(n-k-1)$, with $L$ the likelihood of the model, $k$ its number of consumed degrees of freedom (as returned by dof), and $n$ the number of observations (as returned by nobs).

source
StatsBase.bicFunction
bic(model::StatisticalModel)

Bayesian Information Criterion, defined as $-2 \log L + k \log n$, with $L$ the likelihood of the model, $k$ its number of consumed degrees of freedom (as returned by dof), and $n$ the number of observations (as returned by nobs).

source
StatsBase.coeftableFunction
coeftable(model::StatisticalModel; level::Real=0.95)

Return a table with coefficients and related statistics of the model. level determines the level for confidence intervals (by default, 95%).

The returned CoefTable object implements the Tables.jl interface, and can be converted e.g. to a DataFrame via using DataFrames; DataFrame(coeftable(model)).

source
StatsBase.confintFunction
confint(model::StatisticalModel; level::Real=0.95)

Compute confidence intervals for coefficients, with confidence level level (by default 95%).

source
StatsBase.devianceFunction
deviance(model::StatisticalModel)

Return the deviance of the model relative to a reference, which is usually when applicable the saturated model. It is equal, up to a constant, to $-2 \log L$, with $L$ the likelihood of the model.

source
StatsBase.dofFunction
dof(model::StatisticalModel)

Return the number of degrees of freedom consumed in the model, including when applicable the intercept and the distribution's dispersion parameter.

source
StatsBase.fitFunction
fit(Histogram, data[, weight][, edges]; closed=:left, nbins)

Fit a histogram to data.

Arguments

• data: either a vector (for a 1-dimensional histogram), or a tuple of vectors of equal length (for an n-dimensional histogram).

• weight: an optional AbstractWeights (of the same length as the data vectors), denoting the weight each observation contributes to the bin. If no weight vector is supplied, each observation has weight 1.

• edges: a vector (typically an AbstractRange object), or tuple of vectors, that gives the edges of the bins along each dimension. If no edges are provided, these are determined from the data.

Keyword arguments

• closed: if :left (the default), the bin intervals are left-closed [a,b); if :right, intervals are right-closed (a,b].

• nbins: if no edges argument is supplied, the approximate number of bins to use along each dimension (can be either a single integer, or a tuple of integers).

Examples

# Univariate
h = fit(Histogram, rand(100))
h = fit(Histogram, rand(100), 0:0.1:1.0)
h = fit(Histogram, rand(100), nbins=10)
h = fit(Histogram, rand(100), weights(rand(100)), 0:0.1:1.0)
h = fit(Histogram, [20], 0:20:100)
h = fit(Histogram, [20], 0:20:100, closed=:right)

# Multivariate
h = fit(Histogram, (rand(100),rand(100)))
h = fit(Histogram, (rand(100),rand(100)),nbins=10)
source

Fit a statistical model.

source
fit(ZScoreTransform, X; dims=nothing, center=true, scale=true)

Fit standardization parameters to vector or matrix X and return a ZScoreTransform transformation object.

Keyword arguments

• dims: if 1 fit standardization parameters in column-wise fashion; if 2 fit in row-wise fashion. The default is nothing, which is equivalent to dims=2 with a deprecation warning.

• center: if true (the default) center data so that its mean is zero.

• scale: if true (the default) scale the data so that its variance is equal to one.

Examples

julia> using StatsBase

julia> X = [0.0 -0.5 0.5; 0.0 1.0 2.0]
2×3 Matrix{Float64}:
0.0  -0.5  0.5
0.0   1.0  2.0

julia> dt = fit(ZScoreTransform, X, dims=2)
ZScoreTransform{Float64, Vector{Float64}}(2, 2, [0.0, 1.0], [0.5, 1.0])

julia> StatsBase.transform(dt, X)
2×3 Matrix{Float64}:
0.0  -1.0  1.0
-1.0   0.0  1.0
source
fit(UnitRangeTransform, X; dims=nothing, unit=true)

Fit a scaling parameters to vector or matrix X and return a UnitRangeTransform transformation object.

Keyword arguments

• dims: if 1 fit standardization parameters in column-wise fashion;

if 2 fit in row-wise fashion. The default is nothing.

• unit: if true (the default) shift the minimum data to zero.

Examples

julia> using StatsBase

julia> X = [0.0 -0.5 0.5; 0.0 1.0 2.0]
2×3 Matrix{Float64}:
0.0  -0.5  0.5
0.0   1.0  2.0

julia> dt = fit(UnitRangeTransform, X, dims=2)
UnitRangeTransform{Float64, Vector{Float64}}(2, 2, true, [-0.5, 0.0], [1.0, 0.5])

julia> StatsBase.transform(dt, X)
2×3 Matrix{Float64}:
0.5  0.0  1.0
0.0  0.5  1.0
source
StatsBase.informationmatrixFunction
informationmatrix(model::StatisticalModel; expected::Bool = true)

Return the information matrix of the model. By default the Fisher information matrix is returned, while the observed information matrix can be requested with expected = false.

source
StatsBase.loglikelihoodFunction
loglikelihood(model::StatisticalModel)

Return the log-likelihood of the model.

source
loglikelihood(model::StatisticalModel, ::Colon)

Return a vector of each observation's contribution to the log-likelihood of the model. In other words, this is the vector of the pointwise log-likelihood contributions.

In general, sum(loglikehood(model, :)) == loglikelihood(model).

source
loglikelihood(model::StatisticalModel, observation)

Return the contribution of observation to the log-likelihood of model.

source
StatsBase.nobsFunction
nobs(model::StatisticalModel)

Return the number of independent observations on which the model was fitted. Be careful when using this information, as the definition of an independent observation may vary depending on the model, on the format used to pass the data, on the sampling plan (if specified), etc.

source
StatsBase.nullloglikelihoodFunction
loglikelihood(model::StatisticalModel)

Return the log-likelihood of the null model corresponding to model. This is usually the model containing only the intercept.

source
StatsBase.r2Function
r2(model::StatisticalModel)
r²(model::StatisticalModel)

Coefficient of determination (R-squared).

For a linear model, the R² is defined as $ESS/TSS$, with $ESS$ the explained sum of squares and $TSS$ the total sum of squares.

source
r2(model::StatisticalModel, variant::Symbol)
r²(model::StatisticalModel, variant::Symbol)

Pseudo-coefficient of determination (pseudo R-squared).

For nonlinear models, one of several pseudo R² definitions must be chosen via variant. Supported variants are:

• :MacFadden (a.k.a. likelihood ratio index), defined as $1 - \log (L)/\log (L_0)$;
• :CoxSnell, defined as $1 - (L_0/L)^{2/n}$;
• :Nagelkerke, defined as $(1 - (L_0/L)^{2/n})/(1 - L_0^{2/n})$.
• :devianceratio, defined as $1 - D/D_0$.

In the above formulas, $L$ is the likelihood of the model, $L_0$ is the likelihood of the null model (the model with only an intercept), $D$ is the deviance of the model (from the saturated model), $D_0$ is the deviance of the null model, $n$ is the number of observations (given by nobs).

The Cox-Snell and the deviance ratio variants both match the classical definition of R² for linear models.

source
StatsBase.scoreFunction
score(model::StatisticalModel)

Return the score of the model, that is the gradient of the log-likelihood with respect to the coefficients.

source
StatsBase.vcovFunction
vcov(model::StatisticalModel)

Return the variance-covariance matrix for the coefficients of the model.

source

RegressionModel extends StatisticalModel by implementing the following additional methods.

StatsBase.crossmodelmatrixFunction
crossmodelmatrix(model::RegressionModel)

Return X'X where X is the model matrix of model. This function will return a pre-computed matrix stored in model if possible.

source
StatsBase.predictFunction
predict(model::RegressionModel, [newX])

Form the predicted response of model. An object with new covariate values newX can be supplied, which should have the same type and structure as that used to fit model; e.g. for a GLM it would generally be a DataFrame with the same variable names as the original predictors.

source

An exception type is provided to signal convergence failures during model estimation:

StatsBase.ConvergenceExceptionType
ConvergenceException(iters::Int, lastchange::Real=NaN, tol::Real=NaN)

The fitting procedure failed to converge in iters number of iterations, i.e. the lastchange between the cost of the final and penultimate iteration was greater than specified tolerance tol.

source