Skip to contents

Runs the package's reporting and fit-checking functions over a fitted model and writes the results into one folder, with a markdown index that says what each file is and how to read it. Works for either backend; components a backend or a model does not support are recorded in the index as not produced, rather than stopping the run.

Usage

ctReport(
  fit,
  folder = "ctReport",
  components = "default",
  nsamples = 200,
  cores = getOption("mc.cores", 2L),
  quantiles = c(0.025, 0.5, 0.975),
  times = "auto",
  timeinterval = 1,
  nsubjects = 6,
  lags = 0:3,
  folds = 10,
  profilepoints = 9,
  verbose = TRUE
)

Arguments

fit

A fit from ctFit (class ctStanFit or ctJuliaFit).

folder

Directory to write into; created if it does not exist. Files from a previous ctReport run in the same folder are removed first, so the folder always matches its index.

components

Either one of 'quick' (nothing that needs generated data or further model evaluations), 'default', 'all' (adds cross validation and LaTeX), or a character vector of component names: summary, parmatrices, identification, profile, discretepars, network, predictions, residuals, adequacy (ctFitAdequacy), scoretests (ctScoreTest, julia fits), covcheck, postpred, tipredeffects, crossval, equations.

nsamples

Draws used by the components that sample from the fit's uncertainty, and by data generation for adequacy, covcheck and postpred.

cores

Cores for the components that can use more than one (discretepars, adequacy, scoretests, covcheck, postpred, crossval).

quantiles

Quantiles for the parameter matrices and the discrete parameter plots.

times

Interval grid for discretepars. 'auto' uses zero to the 90th percentile of the observed per-subject time span.

timeinterval

Time interval at which the networks, the discrete-time parameter matrices and the covariate effects are reported.

nsubjects

Number of subjects plotted by predictions, chosen as those with the most observations.

lags

Lags for the lagged covariance check.

folds

Folds for crossval.

profilepoints

Grid points per parameter in the likelihood profile.

verbose

Report each component as it runs.

Value

Invisibly, a list with the folder and, per component, whether it was written, skipped or failed. Called for the files it writes.

Details

The expensive parts are adequacy, covcheck and postpred, which generate data from the fit (once, shared), scoretests, which evaluates the model extended with new terms, and crossval, which refits. components = 'quick' excludes them all.

profile walks the log probability along each raw parameter, holding the others at the estimate, over a grid measured in the standard error that parameter's own curvature implies. It flags parameters whose slice peaks away from the estimate (the optimizer did not arrive), whose surface is not the shape a standard error assumes, and whose reported interval is much wider than their own curvature (they sit on a ridge with other parameters). It is a conditional slice, not a profile likelihood, which is why it is cheap enough to run by default; identification names the ridge directions.

Checking a fit – which function answers what

Can the data inform every parameter?

ctIdentify before fitting; fit$identifiability and ctFitProfile after.

Did the optimiser, or the sampler, get there?

print(fit) says whether it converged, summary(fit) adds R-hat and effective sample sizes for a sampled fit, and ctTracePlot draws the optimiser's trace or each chain's draws.

How far did the data move each parameter from its prior?

ctPlotPosterior.

Does data generated from the fit look like the data?

ctPostPredPlots, and ctFitCheckCov for the lagged covariances, split by a group or predictor if wanted.

Are the one-step-ahead errors what the model says they are?

ctFitAdequacy tests them by family, allowing for the estimation; ctResiduals and ctACFresiduals show them.

What would relaxing the model change?

ctScoreTest, without refitting; ctModelProposals lists the changes and ctModelImprove refits them.

Does a richer model fit better, or predict better?

anova for nested models, ctLOO for out-of-sample prediction.

How much do the julia engine's approximations cost?

ctLaplaceCheck for the Laplace approximation of random effects, ctParticleLik for the filter's Gaussian approximation of the states.

Do the intervals cover as often as they claim?

ctCoverageCheck, by simulation.

Examples

# \donttest{
f <- ctReport(ctstantestfit, folder = file.path(tempdir(), 'ctReport'),
  components = c('summary', 'identification', 'profile'), profilepoints = 5)
list.files(f$folder)
# }