Skip to contents

The filter ctsem fits with is an assumed-density approximation: it carries a Gaussian for the latent states, linearises the drift over each prediction step, and projects non-Gaussian measurement updates back onto a Gaussian. For a linear model with Gaussian indicators it is exact; for anything else the size of its error is invisible from inside it. This function estimates the same marginal log likelihood at the fit's estimates with a bootstrap particle filter that makes none of those approximations, and reports the difference, in total and per row.

Usage

ctParticleLik(
  fit,
  particles = 2000,
  substeps = 20,
  transition = c("exponential", "euler"),
  seed = 1,
  replicates = 1,
  cores = getOption("mc.cores", 2L)
)

Arguments

fit

A fit from ctFit(..., backend = 'julia') with the default intoverpop; random effects carried as augmented states are integrated by the particles. Laplace-route fits are refused.

particles

Particles per subject.

substeps

Transition steps per observation interval. This is the reference mesh, independent of anything the fit used, and floored by the fit's own maxtimestep or automatic mesh.

transition

'exponential' applies the filter's own discretisation at each particle's state: exact for the linear parts of the model, second order otherwise, so the comparison isolates the filter's Gaussian assumption. 'euler' is plain Euler-Maruyama, which shares no approximation with the filter at all but needs several times more substeps.

seed

Fixes every draw; the result is reproducible given it.

replicates

Independent runs (with seeds seed, seed + 1, ...). One run reports an approximate standard error from the effective sample sizes; more than one reports the standard deviation across runs.

cores

Engine threads. The particle filter splits the subjects across them, each subject with its own random stream, so the result does not depend on the count.

Value

A list: loglik, the particle estimate (mean over replicates); se; fit_loglik, the filter's log likelihood from the fit; difference, particle minus filter; rows, a data frame with the subject, time, particle and filter log likelihood of every row and their difference; replicates, the individual estimates; ess_min, the smallest effective sample size any row saw; and the settings.

Details

Each particle steps through the process with drift, diffusion and every state-dependent parameter evaluated at its own state, is weighted by the conditional density of the row's observations given that state (Gaussian, binary, ordinal, count and censored indicators are all handled), and is resampled when the weights degenerate. The estimate is unbiased with Monte Carlo error falling as one over the square root of particles.

It is a diagnostic at fixed parameters, not an estimator. Two things decide whether to trust the comparison: se should be well below the difference you care about (raise particles, or set replicates for an empirical standard error), and doubling substeps should not move the answer.

See also

ctFitUncertainty with method = 'is' and filter = 'particle' applies the correction this measures, to the fit's posterior draws.

Examples

# \donttest{
# Needs a working julia backend, so the example is inert where Julia is
# absent -- including on CRAN, whose check machines have none.
if (isTRUE(ctJuliaStatus()$available)) {
  model <- ctModel(type = 'ct', manifestNames = 'Y1', latentNames = 'eta1',
    LAMBDA = matrix(1))
  model$pars$indvarying <- FALSE
  fit <- ctFit(ctstantestdat, model, backend = 'julia', cores = 1)
  # A linear Gaussian model, where the filter is exact: the difference
  # should be within a few standard errors of zero.
  check <- ctParticleLik(fit, particles = 1000, substeps = 5)
  check[c('loglik', 'se', 'fit_loglik', 'difference')]
  # Julia keeps running between calls, holding what it has compiled;
  # this ends it.
  ctJuliaKill("session")
}
# }