LogExpFunctions

xlogx(x)

Return x * log(x) for x ≥ 0, handling $x = 0$ by taking the downward limit.

julia> xlogx(0)
0.0

xlogy(x, y)

Return x * log(y) for y > 0 with correct limit at $x = 0$.

julia> xlogy(0, 0)
0.0

xlog1py(x, y)

Return x * log(1 + y) for y ≥ -1 with correct limit at $x = 0$.

julia> xlog1py(0, -1)
0.0

xexpx(x)

Return x * exp(x) for x > -Inf, or zero if x == -Inf.

julia> xexpx(-Inf)
0.0

xexpy(x, y)

Return x * exp(y) for y > -Inf, or zero if y == -Inf or if x == 0 and y is finite.

julia> xexpy(1.0, -Inf)
0.0

logistic(x)

The logistic sigmoid function mapping a real number to a value in the interval $[0,1]$,

\[\sigma(x) = \frac{1}{e^{-x} + 1} = \frac{e^x}{1+e^x}.\]

Its inverse is the logit function.

logit(x)

The logit or log-odds transformation, defined as

\[\operatorname{logit}(x) = \log\left(\frac{x}{1-x}\right)\]

for $0 < x < 1$.

Its inverse is the logistic function.

logcosh(x)

Return log(cosh(x)), carefully evaluated without intermediate calculation of cosh(x).

The implementation ensures logcosh(-x) = logcosh(x).

logabssinh(x)

Return log(abs(sinh(x))), carefully evaluated without intermediate calculation of sinh(x).

The implementation ensures logabssinh(-x) = logabssinh(x).

logabstanh(x)

Return log(abs(tanh(x))), evaluated carefully.

The implementation ensures logabstanh(-x) = logabstanh(x).

log1psq(x)

Return log(1+x^2) evaluated carefully for abs(x) very small or very large.

log1pexp(x)

Return log(1+exp(x)) evaluated carefully for largish x.

This is also called the "softplus" transformation, being a smooth approximation to max(0,x). Its inverse is logexpm1.

This is also called the "softplus" transformation (in its default parametrization, see softplus), being a smooth approximation to max(0,x).

See:

• Martin Maechler (2012) “Accurately Computing log(1 − exp(− |a|))”

softplus(x)

The generalized softplus function (Wiemann et al., 2024) takes an additional optional parameter a that control the approximation error with respect to the linear spline. It defaults to a=1.0, in which case the softplus is equivalent to log1pexp.

See:

• Wiemann, P. F., Kneib, T., & Hambuckers, J. (2024). Using the softplus function to construct alternative link functions in generalized linear models and beyond. Statistical Papers, 65(5), 3155-3180.

invsoftplus(y)

The inverse generalized softplus function (Wiemann et al., 2024). See softplus.

log1mexp(x)

Return log(1 - exp(x))

See:

• Martin Maechler (2012) “Accurately Computing log(1 − exp(− |a|))”

Note: different than Maechler (2012), no negation inside parentheses

log2mexp(x)

Return log(2 - exp(x)) evaluated carefully.

logexpm1(x)

Return log(exp(x) - 1) or the “invsoftplus” function. It is the inverse of log1pexp (aka “softplus”).

log1pmx(x)

Return log(1 + x) - x.

Use naive calculation or range reduction outside kernel range. Accurate ~2ulps for all x. This will fall back to the naive calculation for argument types different from Float32, Float64.

logmxp1(x)

Return log(x) - x + 1 carefully evaluated. This will fall back to the naive calculation for argument types different from Float64.

logaddexp(x, y)

Return log(exp(x) + exp(y)), avoiding intermediate overflow/undeflow, and handling non-finite values.

logsubexp(x, y)

Return log(abs(exp(x) - exp(y))), preserving numerical accuracy.

logsumexp(X)

Compute log(sum(exp, X)).

X should be an iterator of real or complex numbers. The result is computed in a numerically stable way that avoids intermediate over- and underflow, using a single pass over the data.

See also logsumexp!.

References

Sebastian Nowozin: Streaming Log-sum-exp Computation

logsumexp(X; dims)

Compute log.(sum(exp.(X); dims=dims)).

The result is computed in a numerically stable way that avoids intermediate over- and underflow, using a single pass over the data.

See also logsumexp!.

References

Sebastian Nowozin: Streaming Log-sum-exp Computation

logsumexp!(out, X)

Compute logsumexp of X over the singleton dimensions of out, and write results to out.

The result is computed in a numerically stable way that avoids intermediate over- and underflow, using a single pass over the data.

See also logsumexp.

References

Sebastian Nowozin: Streaming Log-sum-exp Computation

softmax!(r::AbstractArray{<:Real}, x::AbstractArray{<:Real}=r; dims=:)

Overwrite r with the softmax transformation of x over dimension dims.

That is, r is overwritten with exp.(x), normalized to sum to 1 over the given dimensions.

See also: softmax

softmax(x::AbstractArray{<:Real}; dims=:)

Return the softmax transformation of x over dimension dims.

That is, return exp.(x), normalized to sum to 1 over the given dimensions.

See also: softmax!

cloglog(x)

Compute the complementary log-log, log(-log(1 - x)).

cexpexp(x)

Compute the complementary double exponential, 1 - exp(-exp(x)).

loglogistic(x)

Return log(logistic(x)), computed more carefully and with fewer calls than the naive composition of functions.

Its inverse is the logitexp function.

logitexp(x)

Return logit(exp(x)), computed more carefully and with fewer calls than the naive composition of functions.

Its inverse is the loglogistic function.

log1mlogistic(x)

Return log(1 - logistic(x)), computed more carefully and with fewer calls than the naive composition of functions.

Its inverse is the logit1mexp function.

logit1mexp(x)

Return logit(1 - exp(x)), computed more carefully and with fewer calls than the naive composition of functions.

Its inverse is the log1mlogistic function.