HMC-Laplace

HMC-Laplace is Latte's sampling engine. It runs the No-U-Turn Sampler (NUTS) over the hyperparameters, with the latent field marginalised out by a Laplace approximation at every step. The entry point is hmc_laplace.

How it works

This is not plain NUTS over the whole model. Sampling the latent field and the hyperparameters together is the funnel-shaped problem that makes Hamiltonian Monte Carlo struggle on hierarchical models. HMC-Laplace sidesteps it by letting NUTS move only in the low-dimensional hyperparameter space $θ$ .

The price is that, to evaluate its target $p (θ ∣ y)$ , NUTS needs the marginal likelihood $p (y ∣ θ)$ , and that means integrating out the latent field. HMC-Laplace does this the same way INLA and TMB do, with an inner Laplace approximation. At every $θ$ the sampler visits, it solves the latent field's Gaussian approximation, reads off $p (y ∣ θ)$ and its gradient, and hands them back to NUTS. So every step the sampler takes carries an inner Laplace solve. The picture below shows it: each draw on the left runs its own inner Laplace, drawn on the right.

What the cost buys is faithfulness. Because NUTS samples the real $p (θ ∣ y)$ , the hyperparameter posterior comes out with whatever skew it genuinely has, which is exactly where TMB's single Gaussian and INLA's coarse grid can fall short. The latent marginals are the average of the inner Laplace approximations across the draws.

NUTS draws θ from p(θ|y)

→

each draw runs an inner Laplace

draws 16

inner Laplace / drawmarginal (their average)exact

Not just NUTS: every draw solves the latent field at its own θ (one highlighted). Averaging those inner Laplace fits gives the marginal.

Tuning

Chain length (`n_samples`, `n_warmup`)

The usual MCMC controls. n_warmup (default 200) is the adaptation and burn-in, and the kept draws number n_samples (default 500). Raise both when the effective sample size is low or the marginals look noisy. Each step runs an inner Laplace, so longer chains cost real wall-clock time.

julia

hmc_laplace(model, y; n_samples = 2000, n_warmup = 1000)

Reproducibility (`rng`) and gradients (`diff_strategy`)

Pass an rng for a chain you can reproduce. diff_strategy sets how the gradient of the target is taken, as on the TMB page; the default ADStrategy() suits @latte models with recognised GMRF latents.

Reference

Latte.hmc_laplace Function

julia

hmc_laplace(model::LatentGaussianModel, y;
            n_samples=500, n_warmup=200,
            rng=Random.default_rng(),
            progress=false) -> HMCLaplaceResult

tmbstan-style HMC on the Laplace marginal. Samples θ via NUTS with the Laplace approximation q(x | θ) substituted for the true p(x | y, θ); the latent field is reconstructed per-sample from the inner Laplace at each drawn θ.

Pipeline:

Run tmb(model, y) → MAP θ̂, Laplace covariance Σ_θ̂. Used as warm-start initial point and as the HMC mass matrix (dense metric M⁻¹ = Σ_θ̂). Preconditioning matches the target's local curvature, so NUTS tree depths stay low.
NUTS samples θ on L(θ) = log p(y, θ) via hyperparameter_logpdf. Gradients by finite differences (fine for typical small |θ|).
For each sample θ_k, recompute the inner Gaussian approximation q(x | θ_k) and extract conditional means + marginal SDs.
Assemble HMCLaplaceResult with Tier 1 protocol marginals built from chain samples.

Arguments

model: a LatentGaussianModel (e.g., from latte_from_dppl or a hand-built spec).
y: observations. Same handling as inla() / tmb() — _prepare_for_prediction normalises integer vectors into PoissonObservations etc.
n_samples / n_warmup: NUTS post-warmup and warmup steps.
rng: seedable RNG for reproducibility.
progress: pass through to AdvancedHMC.
diff_strategy: forwarded to the TMB warm-start (mode + Σ_θ). Default ADStrategy() is noise-robust on augmented LGMs and works for @latte models with recognized GMRF latents (IID / RW / AR1 / Besag) and for the broad class of custom-logpdf likelihoods. Reach for FiniteDiffStrategy() only in the narrow case where a hyperparameter-derived value is hoisted into the observation payload by the @latte prelude-lift (e.g. φ = exp(log_φ)in Tweedie), which the outer Hessian can't keepDual-typed; tracked intasks/dppl-adapter-outer-ad-closure.org.

Diagnostics

Method-specific accessors on the returned HMCLaplaceResult: samples, divergences, mean_tree_depth, acceptance_rate, mean_step_size. Use these to judge convergence before trusting the marginals.

source

Limits

It is MCMC, so the marginals carry Monte Carlo noise. That noise shrinks with the effective sample size, unlike the deterministic INLA and TMB. Check the ESS and run longer chains before making tail-sensitive claims.
It is the slowest engine. Every leapfrog step is an inner Laplace solve, and NUTS takes many steps per draw. Reach for it when the faithfulness is worth the time, or when the hyperparameter posterior is too awkward for a grid.
Constrained or rank-deficient latents can stall it. NUTS occasionally proposes a $θ$ where the inner solve is numerically singular. Those steps are rejected, which can hurt mixing.

References

NUTS

The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo

M. D. Hoffman & A. Gelman

Journal of Machine Learning Research · 2014 · arXiv:1111.4246

The No-U-Turn Sampler, the adaptive Hamiltonian Monte Carlo algorithm Latte runs over the hyperparameters.

Embedded Laplace + HMC

Hamiltonian Monte Carlo using an Adjoint-differentiated Laplace Approximation

C. C. Margossian, A. Vehtari, D. Simpson & R. Agrawal

Advances in Neural Information Processing Systems (NeurIPS) · 2020 · arXiv:2004.12550

Hamiltonian Monte Carlo over the hyperparameters, with the latent Gaussian field marginalised by an embedded Laplace approximation and the gradient propagated through that inner solve. The method this engine implements.

HMC-Laplace ​

How it works ​

Tuning ​

Chain length (n_samples, n_warmup) ​

Reproducibility (rng) and gradients (diff_strategy) ​

Reference ​

Limits ​

References ​

See also ​