its.analysis: Running Interrupted Time Series Analysis

Description

Two functions for running and then post-estimating an Interrupted Time Series Analysis model. This is a solution for running time series analyses on temporally short data. See English (2019) 'The its.analysis R package - Modelling short time series data' <https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3398189> for an overview of the method.

Details

Interrupted Time Series Analysis (ITSA) serves as a replacement for traditional time series modelling techniques when sample sizes are too small for such approaches (including ARIMA and other OLS-orientated methods).

The ITSA model offered in this package is a Type-II sum-squares ANCOVA Lagged Dependent Variable model. The main function produces a range of additional bolt-ons which provide further detailed information and test assumptions relative to both ANOVA and time series analysis.

The key independent variable must, if it is not already, be converted into a factor whereby periods of significant change in the behaviour or direction of the variable are identified and coded as separate periods (with one numeric identifier for their entire length). This creates the 'interruption framework'.

The ITSA model will automatically calculate and apply the lag of the dependent variable to as a covariate, accounting for the trend in the series itself before estimating the variance accounted for from the interruption. Further covariates may be specified, but caution should be used not to over-fit.

A number of post-estimation checks are ran on the model, including examining residual normality, homogeneous variances, and residual autocorrelation. The results of each test can be seen using the itsa.postest function. If there is potential evidence of violation in the first two assumptions, or if there is evidence of residual autocorrelation in the first three lags, a warning message will appear under the results from the itsa.model function. This warning does not necessarily mean that the model is invalid, but that researchers ought to run and pay careful attention to the results of the post-estimation.

By default, 1000 bootstrap replications of the model are estimated using randomly drawn samples. Returned from this model are the 90 per cent confidence intervals and a trimmed median (10 per cent removed). The bootstrap replications can be visualised using the post-estimation function.

Note: The package will handle time series where only two periods are defined (i.e. a 'normal' (0) period and an 'interrupted' (1) period), or time series with multiple different periods (i.e 'pre', 'during' and 'post' interruption, or perhaps multiple different interruptions). Users should also keep in mind that a minimum of between 7 and 10 cases per interruption variable level should ideally be followed to allow for enough power.

Run Interrupted Time Series Analyses

Description

Sets up an Interrupted Time Series Analysis (ITSA) for analysing short time series data.

Usage

itsa.model(
  data = NULL,
  time = NULL,
  depvar = NULL,
  interrupt_var = NULL,
  covariates = NULL,
  alpha = 0.05,
  no.plots = FALSE,
  bootstrap = TRUE,
  Reps = 1000,
  parr = "no",
  print = TRUE
)

Arguments

Details

This function provides an Interrupted Time Series Analysis (ITSA) model using a variance-centric approach to estimating change in a dependent variable over time.

Using the defined variables, a Type-2 Sum Squares ANCOVA Lagged Dependent Variable model is fitted which estimates the difference in means between interrupted and non-interrupted time periods, while accounting for the lag of the dependent variable and any further specified covariates.

The function includes a bootstrap model, which runs by default. This repeats 1000 replications of the main model with randomly drawn samples. A trimmed median (10 percent removed) F-value is reported and a bootstrapped p-value derived from it. Users may turn off the bootstrapping model, or the number of replications for the bootstrap model to pass through can be altered using the Reps argument.

Variable names must be defined using quotation marks, or their column index number within the data. Any number of covariates can be passed into the covariates argument as a list of vectors (both must be within the same data as the dependent and independent variables).

Also returned as warning messages are the results of ANOVA and residual autocorrelation assumptions check ran in the background, if any are potentially violated. These tests and further post-estimation can be done through the itsa.postest function.

Returns to console the results from analysis of variance test, results of the F-value bootstrap model, and a summary of the result (relative to user defined alpha) and any assumption violation warnings. Users may force this return off by declaring print=FALSE in the model arguments.

Further returns a plot to the plot window graphing the dependent variable time series and interruption points. As this is a ggplot2 generated object, users can call the plot and make further customisations to it as an output.

Assigning to an object will return a list of all of the above, plus further tests including a Tukey Honest Significant Differences test, the data and arguments set by the user, a table of time-series group means, the full bootstrap model results and results of assumptions tests, and the full list of residual and fitted values.

If any of data, depvar, interrupt_var, or time are undefined, the process will stop and an error message will appear.

Value

Returns a large list which can be assigned to the global environment, containing results and necessary information for running post-estimation itsa.postest function. It also contains the Time Series Interruption plot (itsa.plot) and bootstrap results, if applied.

Examples

# Build variables

year <- c(2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008,
2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018)
depv <- c(8.22, 8.19, 8.23, 8.28, 8.32, 8.39, 8.02,
7.92, 7.62, 7.23, 7.1, 7.11, 6.95, 7.36, 7.51, 7.78, 7.92, 7.81)
interruption <- c(0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0)
cov1 <- c(3.1, 3.3, 5.1, 5.2, 5.4, 4.5, 4.7, 4.9, 5.3,
5.6, 5.8, 6.0, 4.8, 5.2, 4.5, 4.6, 5.1, 4.7)

x <- as.data.frame(cbind(year, depv, interruption, cov1))

# First example model
itsa.model(data=x, time="year", depvar="depv", interrupt_var = "interruption",
alpha=0.05, bootstrap=TRUE, Reps = 250)

# Add a covariate, change alpha to 0.1
itsa.model(data=x, time="year", depvar="depv", interrupt_var = "interruption",
covariates = "cov1", alpha=0.1, bootstrap=TRUE, Reps = 250)

# Force no plots and turn off bootstrap
itsa.model(data=x, time="year", depvar="depv", interrupt_var = "interruption",
covariates = "cov1", alpha=0.1, no.plots=TRUE, bootstrap=FALSE)

# Example of no significant result
itsa.model(data=x, time="year", depvar="cov1", interrupt_var = "interruption",
 alpha=0.05, bootstrap=FALSE)

Post-estimating ITSA models

Description

Function to produce post-estimation plots from the itsa.model function.

Usage

itsa.postest(
  model = NULL,
  no.plots = FALSE,
  alpha = 0.05,
  bootstrap = TRUE,
  print = TRUE
)

Arguments

Details

This function runs and reports post-estimation tests on fits from the itsa.model function, and generates four plots.

Main tests are whether two key ANCOVA assumptions are met, and an additional autocorrelation test for the time series framework.

The Shapiro-Wilks test examines the residuals from the fitted model for abnormality. A p-value less than alpha indicates abnormal residuals. In this instance, the user should consider replicating the its.analysis model design using a non-parametric ANCOVA equivalent.

The Levene's Test makes sure that there are equal variances between the treated groups. A p-value less than alpha indicates heterogeneous variances.

A QQ-Norm and Boxplot are generated with the test results overlaid (respectively), with a Residual v Fitted and Autocorrelation Function Plot also generated.

The results of bootstrap estimations in itsa.model will be plotted, unless argument is switched to FALSE. See plot.boot help file for assistance on interpreting these plots.

Default is to generate plots and summary table, but plots may be overridden using no.plots argument.

Default alpha value for post-estimation statistics is 0.05, test results will suggest potential presence of problems at higher values (and also at higher levels relative to a user-defined alpha), but user discretion is needed (examined in tandem with the Residuals v Fitted plot).

See 'itsa.model' documentation for further information.

Value

Returns a list containing post-estimation test results and four plots (plus one additional if bootstrapping applied) which are also sent directly to the plot tab.