Hyperparameter Posterior

The hyperparameter posterior module implements the INLA approximation to the hyperparameter posterior π(θ | y), including mode finding, posterior exploration, and marginalization.

Overview

INLA approximates the hyperparameter posterior using the Laplace approximation around the mode θ*. The hyperparameter posterior system provides:

• Mode finding using optimization methods to find θ* = argmax π(θ | y)

• Posterior exploration around the mode via grid or CCD design

• Hyperparameter marginalization via spline-based strategies for O(1) queries

• Numerical stability through proper handling of log-space computations

Basic Usage

Mode Finding

using Latte
using Distributions
using GaussianMarkovRandomFields

# Set up model with @hyperparams macro
spec = @hyperparams begin
    (σ ~ InverseGamma(2, 1), transform = log, space = natural)
end

function latent_gmrf(; σ, kwargs...)
    return (zeros(5), spdiagm(0 => fill(1/σ^2, 5)))  # (mean, precision)
end

obs_model = ExponentialFamily(Normal)
model = LatentGaussianModel(spec, FunctionLatentModel(latent_gmrf, 5), obs_model)

# Observed data
y = [0.5, -0.2, 0.8, -0.1, 0.3]

# Find hyperparameter mode
θ_star, mode_points, mode_logdensities = find_hyperparameter_mode(model, y)

Full Inference

# Run full INLA (mode finding + exploration + marginalization)
result = inla(model, y)

# Access hyperparameter marginals
summary_df(result.hyperparameter_marginals)

# Access latent marginals
mean.(result.latent_marginals)

Custom Exploration and Marginalization

# Use CCD exploration with custom scaling
result = inla(model, y,
    exploration_strategy = CCDExplorationStrategy(f0 = 1.3)
)

# Use grid exploration with custom parameters
result = inla(model, y,
    exploration_strategy = GridExplorationStrategy(
        max_log_drop = 3.0,
        interpolation_subdivisions = 2
    )
)

Mathematical Background

INLA Hyperparameter Approximation

The INLA approach approximates the hyperparameter posterior as:

$$\tilde{\pi }(\theta |y)\propto \frac{\pi (x^{\ast }(\theta ),\theta ,y)}{{\tilde{\pi }}_G(x^{\ast }(\theta )|\theta ,y)}$$

where:

• x*(θ) is the mode of the latent field for given θ

• π̃_G is the Gaussian approximation to the latent field posterior

Posterior Exploration

The exploration builds grids for integration by:

1. Reparameterization: Transform θ → z using eigendecomposition of the Hessian

2. Design point placement: Grid (Cartesian product) or CCD (Central Composite Design)

3. Evaluation: Compute log-density and latent marginals at each design point

4. Normalization: Normalize using integration weights

Hyperparameter Marginalization

1D marginal posteriors are computed by profiling or summation, then fit with cubic splines:

$$\tilde{\pi }({\theta }_j|y)=\int \tilde{\pi }(\theta |y)d{\theta }_{-j}$$

All downstream queries (logpdf, cdf, quantile) are O(1) spline lookups.

API Reference

Mode Finding

Latte.hyperparameter_logpdf Function

hyperparameter_logpdf(model::LatentGaussianModel, θ, y, ga=nothing)

Evaluate log π(θ | y) ∝ log π(θ) + log π(x_(θ), θ, y) - log π̃_G(x_(θ) | θ, y)

This is the INLA approximation to the hyperparameter posterior.

Arguments

• model::LatentGaussianModel: The INLA model specification

• θ: Hyperparameters (WorkingHyperparameters or NaturalHyperparameters)

• y: Observed data

• ga: Optional pre-computed Gaussian approximation (GMRF object). If nothing, will be computed.

Details

• Main implementation is for WorkingHyperparameters (working space)

• NaturalHyperparameters converts to working space and adds Jacobian correction

source

Latte.find_hyperparameter_mode Function

find_hyperparameter_mode(model::LatentGaussianModel, y; method=BFGS(), collect_points=true, progress_callback=nothing)

Find the mode θ* of the hyperparameter posterior π(θ | y).

Arguments

• model: INLA model specification

• y: Observed data

• method: Optimization method (from Optim.jl)

• collect_points: Whether to collect intermediate points during optimization

• progress_callback: Optional function for progress updates with signature f(; kwargs...)

Returns

• θ_star: The posterior mode in working space (WorkingHyperparameters)

• mode_points: WorkingHyperparameters evaluated during optimization (if collect_points=true)

• mode_logdensities: Log-densities at mode_points (if collect_points=true)

Details

Optimization is performed in working (unconstrained) space. The mode is returned in working space.

source

Exploration

Latte.explore_hyperparameter_posterior Function

explore_hyperparameter_posterior(strategy, model, y, θ_star, marginalization_method, marginalization_indices; kwargs...)

Explore the hyperparameter posterior around the mode θ_star.

Dispatches on the strategy argument:

• GridExplorationStrategy: Cartesian grid exploration

• CCDExplorationStrategy: Central Composite Design exploration

• AutoExplorationStrategy: Automatic selection (grid for D ≤ 2, CCD for D ≥ 3)

Arguments

• strategy::ExplorationStrategy: Exploration strategy (controls grid layout and parameters)

• model::LatentGaussianModel: The INLA model

• y: Observed data

• θ_star::WorkingHyperparameters: The posterior mode in working space

• marginalization_method: Method for latent marginalization at each grid point

• marginalization_indices: Indices of latent field to marginalize

Keyword Arguments

• progress_callback: Optional function for progress updates with signature f(; kwargs...)

• accumulators::Tuple = (): Tuple of PosteriorAccumulator objects to process integration points

Returns

• AbstractHyperparameterExploration: Exploration results (GridExploration or CCDExploration)

• accumulators: Tuple of finalized accumulators (if provided)

source

explore_hyperparameter_posterior(::CCDExplorationStrategy, model, y, θ_star, ...)

CCD-based exploration of the hyperparameter posterior (Rue et al. 2009, Section 6.5).

Uses O(2d² + 1) design points instead of a full Cartesian grid (O(m^d)), enabling models with 3+ hyperparameters. Integration weights are computed analytically (Rue et al. 2009, Section 6.5), which is exact when the posterior in z-space is standard Gaussian.

source