Skip to contents

Compares data generated from the fitted model against the data it was fitted to, as a set of diagnostic plots. Works for both the stan and julia backends. ctPostPredict and ctStanPostPredict are maintained as aliases.

Usage

ctPostPredPlots(
  fit,
  panels = "all",
  variables = NULL,
  nsamples = NA,
  datarows = "all",
  diffsize = 1,
  interval = 0.95,
  resolution = 100,
  jitter = 0,
  residuals = FALSE,
  timebins = 10,
  lags = 0:5,
  lagcor = TRUE,
  notes = TRUE,
  plot = FALSE,
  wait = FALSE
)

Arguments

fit

A fitted ctsem model.

panels

Which panels to produce. 'all' (the default), 'structure', 'calibration', or a character vector of panel names – see Details.

variables

Character vector of variable names to restrict to, or NULL for all. 'LogLik' is a valid name here.

nsamples

Integer or NA. Number of generated datasets to compare against. NA uses whatever the fit already carries.

datarows

Integer vector of data rows to use, or 'all'.

diffsize

Integer vector > 0. Lags, in observations, for the rate-of-change panels. One pair of panels is produced per lag.

interval

Numeric in (0,1). Width of the central predictive interval used by the PredictedVsObserved and IntervalCoverage panels.

resolution

Positive integer. Grid resolution of the shaded model density in the bivariate panels.

jitter

Non-negative numeric. Jitter added to the values in the rate-of-change panels, as a proportion of each variable's standard deviation, applied to observed and generated alike. Useful when data are rounded or ordinal. 0, the default, adds none.

residuals

Logical. Include standardised prior residuals as extra variables. Slow – it needs one Kalman pass per generated dataset.

timebins

Positive integer. Number of equal-count bins of time for the MeanTrajectory panel. Subjects are rarely observed at identical times, so the mean at a time is a mean within a bin of times.

lags

Non-negative integer vector. Observation lags for the LaggedCovariance panel, passed to ctFitCheckCov.

lagcor

Logical. If TRUE (the default) the LaggedCovariance panel shows correlations, which are usually easier to read; FALSE shows covariances.

notes

Logical. If TRUE (the default), each panel carries a caption saying what it shows and what a departure from its reference line means. Set FALSE for bare plots.

plot

Logical. If TRUE, prints the panels and returns them invisibly. If FALSE (the default) returns the list without printing.

wait

Logical. If TRUE and plot=TRUE, waits for a keypress between panels.

Value

A named list of ggplot objects, invisibly if plot=TRUE.

Details

Two families of panel are produced.

structure asks whether the model reproduces the shape of the data: Density (marginal distributions), MeanTrajectory (the mean over subjects against time), ValueByTime and ValueByOccasion (the model-implied density of each variable against time and against occasion number, with the observed data as points on top), ChangeByValue and ChangeByTime (rate of change against current value and against time), and LaggedCovariance (the lagged covariance or correlation structure). ChangeByValue is the phase plane, and is the panel that shows nonlinear dynamics: a linear model implies a straight band with a constant slope, so curvature in the observed points relative to the model shading is evidence the drift is not linear.

MeanTrajectory and ValueByTime both plot against time and answer different questions. ValueByTime shows where the model puts an individual observation, so its envelope is as wide as the data; MeanTrajectory shows where it puts the mean of a bin of observations, whose sampling distribution is narrower by roughly the square root of the bin count. A bias in the average level is therefore visible in the second while sitting comfortably inside the first.

LaggedCovariance is ctFitCheckCov rendered as a dashboard panel: one plot per variable, faceted by the variable it is paired with, empirical against the central 95% of the generated datasets at each lag. It is the only panel that looks at dependence across time rather than at each time, which is what makes it the one that responds to a wrong drift or diffusion when every marginal already matches. Call ctFitCheckCov directly to split it by a group or a predictor.

calibration asks whether the predictive distribution is the right width and in the right place: PredictedVsObserved, IntervalCoverage, PIT, and the PIT resolved by CalibrationByInterval and CalibrationBySubject. A departure that depends on the time interval implicates the dynamics rather than the measurement model, since the measurement model cannot know the interval.

The row-wise log likelihood appears throughout as an extra variable, LogLik – its distribution should match between model and data like any other quantity. It is omitted from the rate-of-change panels, where the derivative of a log likelihood with respect to time is not a meaningful quantity. Note that the observed side of the LogLik comparison is evaluated at the point estimate, so it carries no parameter uncertainty while the generated side does.

SubjectLogLik asks the same question one level up: each subject's total log likelihood, summed over that subject's rows, compared as a cumulative distribution across subjects against the same curve from each generated dataset. Aggregating within a subject first is what makes a subject the model fits badly across many occasions show up, rather than being spread thin across the row-wise panels. The comparison is against replicate datasets of the same size rather than a smooth density, so it is the right reference however few subjects there are.

The approximation is only as good as the number of generated datasets, so a fit carrying few draws gives a coarse picture – the PIT in particular is discrete on nsamples + 1 values.

See also

ctPostPredData for the underlying table, ctFitCheckCov for the lagged covariance panel on its own, with splits by group or predictor, and ctPhasePortrait for the model-implied phase plane on its own.

Examples

# \donttest{
plots <- ctPostPredPlots(ctstantestfit)
print(plots$PIT)
# }