Compares data generated from the fitted model against the data it was fitted to, as a
set of diagnostic plots. Works for both the stan and julia backends.
ctPostPredict and ctStanPostPredict are maintained as aliases.
Usage
ctPostPredPlots(
fit,
panels = "all",
variables = NULL,
nsamples = NA,
datarows = "all",
diffsize = 1,
interval = 0.95,
resolution = 100,
jitter = 0,
residuals = FALSE,
timebins = 10,
lags = 0:5,
lagcor = TRUE,
notes = TRUE,
plot = FALSE,
wait = FALSE
)Arguments
- fit
A fitted ctsem model.
- panels
Which panels to produce.
'all'(the default),'structure','calibration', or a character vector of panel names – see Details.- variables
Character vector of variable names to restrict to, or NULL for all.
'LogLik'is a valid name here.- nsamples
Integer or NA. Number of generated datasets to compare against. NA uses whatever the fit already carries.
- datarows
Integer vector of data rows to use, or
'all'.- diffsize
Integer vector > 0. Lags, in observations, for the rate-of-change panels. One pair of panels is produced per lag.
- interval
Numeric in (0,1). Width of the central predictive interval used by the
PredictedVsObservedandIntervalCoveragepanels.- resolution
Positive integer. Grid resolution of the shaded model density in the bivariate panels.
- jitter
Non-negative numeric. Jitter added to the values in the rate-of-change panels, as a proportion of each variable's standard deviation, applied to observed and generated alike. Useful when data are rounded or ordinal. 0, the default, adds none.
- residuals
Logical. Include standardised prior residuals as extra variables. Slow – it needs one Kalman pass per generated dataset.
- timebins
Positive integer. Number of equal-count bins of time for the
MeanTrajectorypanel. Subjects are rarely observed at identical times, so the mean at a time is a mean within a bin of times.- lags
Non-negative integer vector. Observation lags for the
LaggedCovariancepanel, passed toctFitCheckCov.- lagcor
Logical. If TRUE (the default) the
LaggedCovariancepanel shows correlations, which are usually easier to read; FALSE shows covariances.- notes
Logical. If TRUE (the default), each panel carries a caption saying what it shows and what a departure from its reference line means. Set FALSE for bare plots.
- plot
Logical. If TRUE, prints the panels and returns them invisibly. If FALSE (the default) returns the list without printing.
- wait
Logical. If TRUE and
plot=TRUE, waits for a keypress between panels.
Details
Two families of panel are produced.
structure asks whether the model reproduces the shape of the data:
Density (marginal distributions), MeanTrajectory (the mean over subjects
against time), ValueByTime and ValueByOccasion (the model-implied density
of each variable against time and against occasion number, with the observed data as
points on top), ChangeByValue and ChangeByTime (rate of change against
current value and against time), and LaggedCovariance (the lagged covariance or
correlation structure). ChangeByValue is the phase plane, and is the panel that
shows nonlinear dynamics: a linear model implies a straight band with a constant slope, so
curvature in the observed points relative to the model shading is evidence the drift is
not linear.
MeanTrajectory and ValueByTime both plot against time and answer different
questions. ValueByTime shows where the model puts an individual observation, so its
envelope is as wide as the data; MeanTrajectory shows where it puts the mean
of a bin of observations, whose sampling distribution is narrower by roughly the square
root of the bin count. A bias in the average level is therefore visible in the second while
sitting comfortably inside the first.
LaggedCovariance is ctFitCheckCov rendered as a dashboard panel: one
plot per variable, faceted by the variable it is paired with, empirical against the central
95% of the generated datasets at each lag. It is the only panel that looks at dependence
across time rather than at each time, which is what makes it the one that responds
to a wrong drift or diffusion when every marginal already matches. Call
ctFitCheckCov directly to split it by a group or a predictor.
calibration asks whether the predictive distribution is the right width and in
the right place: PredictedVsObserved, IntervalCoverage, PIT, and the
PIT resolved by CalibrationByInterval and CalibrationBySubject. A departure
that depends on the time interval implicates the dynamics rather than the measurement
model, since the measurement model cannot know the interval.
The row-wise log likelihood appears throughout as an extra variable, LogLik –
its distribution should match between model and data like any other quantity. It is
omitted from the rate-of-change panels, where the derivative of a log likelihood with
respect to time is not a meaningful quantity. Note that the observed side of the LogLik
comparison is evaluated at the point estimate, so it carries no parameter uncertainty
while the generated side does.
SubjectLogLik asks the same question one level up: each subject's total log
likelihood, summed over that subject's rows, compared as a cumulative distribution
across subjects against the same curve from each generated dataset. Aggregating within
a subject first is what makes a subject the model fits badly across many occasions show
up, rather than being spread thin across the row-wise panels. The comparison is against
replicate datasets of the same size rather than a smooth density, so it is the right
reference however few subjects there are.
The approximation is only as good as the number of generated datasets, so a fit carrying
few draws gives a coarse picture – the PIT in particular is discrete on
nsamples + 1 values.
See also
ctPostPredData for the underlying table,
ctFitCheckCov for the lagged covariance panel on its own, with splits by
group or predictor, and ctPhasePortrait for the model-implied phase plane
on its own.
Examples
# \donttest{
plots <- ctPostPredPlots(ctstantestfit)
print(plots$PIT)
# }