imputeFin: Imputation of Financial Time Series with Missing Values.

Description

Missing values often occur in financial data due to a variety of reasons (errors in the collection process or in the processing stage, lack of asset liquidity, lack of reporting of funds, etc.). However, most data analysis methods expect complete data and cannot be employed with missing values. One convenient way to deal with this issue without having to redesign the data analysis method is to impute the missing values. This package provides an efficient way to impute the missing values based on modeling the time series with a random walk or an autoregressive (AR) model, convenient to model log-prices and log-volumes in financial data. In the current version, the imputation is univariate-based (so no asset correlation is used). In addition, outliers can be detected and removed.

Functions

fit_AR1_Gaussian, impute_AR1_Gaussian, fit_AR1_t, impute_AR1_t, plot_imputed

Data

ts_AR1_Gaussian, ts_AR1_t

Help

For a quick help see the README file: GitHub-README.

For more details see the vignette: CRAN-vignette.

Author(s)

Junyan Liu, Rui Zhou, and Daniel P. Palomar

References

J. Liu, S. Kumar, and D. P. Palomar, "Parameter estimation of heavy-tailed AR model with missing data via stochastic EM," IEEE Trans. on Signal Processing, vol. 67, no. 8, pp. 2159-2172, Apr. 2019. <https://doi.org/10.1109/TSP.2019.2899816>

R. Zhou, J. Liu, S. Kumar, and D. P. Palomar, "Student’s t VAR Modeling with Missing Data via Stochastic EM and Gibbs Sampling," IEEE Trans. on Signal Processing, vol. 68, pp. 6198-6211, Oct. 2020. <https://doi.org/10.1109/TSP.2020.3033378>

Fit Gaussian AR(1) model to time series with missing values and/or outliers

Description

Estimate the parameters of a univariate Gaussian AR(1) model to fit the given time series with missing values and/or outliers. For multivariate time series, the function will perform a number of individual univariate fittings without attempting to model the correlations among the time series. If the time series does not contain missing values, the maximum likelihood (ML) estimation is done in one shot. With missing values, the iterative EM algorithm is employed for the estimation until converge is achieved.

Usage

fit_AR1_Gaussian(
  y,
  random_walk = FALSE,
  zero_mean = FALSE,
  remove_outliers = FALSE,
  outlier_prob_th = 0.001,
  verbose = TRUE,
  return_iterates = FALSE,
  return_condMeanCov = FALSE,
  tol = 1e-08,
  maxiter = 100
)

Arguments

Value

If the argument y is a univariate time series (i.e., coercible to a numeric vector), then this function will return a list with the following elements:

If the argument y is a multivariate time series (i.e., with multiple columns and coercible to a numeric matrix), then this function will return a list with each element as in the case of univariate y corresponding to each of the columns (i.e., one list element per column of y), with the following additional elements that combine the estimated values in a convenient vector form:

Author(s)

Junyan Liu and Daniel P. Palomar

References

R. J. Little and D. B. Rubin, Statistical Analysis with Missing Data, 2nd ed. Hoboken, N.J.: John Wiley & Sons, 2002.

J. Liu, S. Kumar, and D. P. Palomar, "Parameter estimation of heavy-tailed AR model with missing data via stochastic EM," IEEE Trans. on Signal Processing, vol. 67, no. 8, pp. 2159-2172, 15 April, 2019.

See Also

impute_AR1_Gaussian, fit_AR1_t

Examples

library(imputeFin)
data(ts_AR1_Gaussian)
y_missing <- ts_AR1_Gaussian$y_missing
fitted <- fit_AR1_Gaussian(y_missing)

Fit Student's t AR(1) model to time series with missing values and/or outliers

Description

Estimate the parameters of a univariate Student's t AR(1) model to fit the given time series with missing values and/or outliers. For multivariate time series, the function will perform a number of indidivual univariate fittings without attempting to model the correlations among the time series. If the time series does not contain missing values, the maximum likelihood (ML) estimation is done via the iterative EM algorithm until converge is achieved. With missing values, the stochastic EM algorithm is employed for the estimation (currently the maximum number of iterations will be executed without attempting to check early converge).

Usage

fit_AR1_t(
  y,
  random_walk = FALSE,
  zero_mean = FALSE,
  fast_and_heuristic = TRUE,
  remove_outliers = FALSE,
  outlier_prob_th = 0.001,
  verbose = TRUE,
  return_iterates = FALSE,
  return_condMean_Gaussian = FALSE,
  tol = 1e-08,
  maxiter = 100,
  n_chain = 10,
  n_thin = 1,
  K = 30
)

Arguments

Value

If the argument y is a univariate time series (i.e., coercible to a numeric vector), then this function will return a list with the following elements:

If the argument y is a multivariate time series (i.e., with multiple columns and coercible to a numeric matrix), then this function will return a list with each element as in the case of univariate y corresponding to each of the columns (i.e., one list element per column of y), with the following additional elements that combine the estimated values in a convenient vector form:

Author(s)

Junyan Liu and Daniel P. Palomar

References

J. Liu, S. Kumar, and D. P. Palomar, "Parameter estimation of heavy-tailed AR model with missing data via stochastic EM," IEEE Trans. on Signal Processing, vol. 67, no. 8, pp. 2159-2172, 15 April, 2019.

See Also

impute_AR1_t, fit_AR1_Gaussian, fit_VAR_t

Examples

library(imputeFin)
data(ts_AR1_t) 
y_missing <- ts_AR1_t$y_missing
fitted <- fit_AR1_t(y_missing)

Fit Student's t VAR model to time series with missing values and/or outliers

Description

Estimate the parameters of a Student's t vector autoregressive model

y_t = \phi_0 + \sum_{i=1}^p \Phi_i * y_{t-i} + \epsilon_t

to fit the given time series with missing values. If the time series does not contain missing values, the maximum likelihood (ML) estimation is done via the iterative EM algorithm until converge is achieved. With missing values, the stochastic EM algorithm is employed for the estimation (currently the maximum number of iterations will be executed without attempting to check early converge).

Usage

fit_VAR_t(
  Y,
  p = 1,
  omit_missing = FALSE,
  parallel_max_cores = max(1, parallel::detectCores() - 1),
  verbose = FALSE,
  return_iterates = FALSE,
  initial = NULL,
  L = 10,
  maxiter = 50,
  ptol = 0.001,
  partition_groups = TRUE,
  K = round(maxiter/3)
)

Arguments

Value

A list with the following elements:

Author(s)

Rui Zhou and Daniel P. Palomar

References

R. Zhou, J. Liu, S. Kumar, and D. P. Palomar, "Student’s t VAR Modeling with Missing Data via Stochastic EM and Gibbs Sampling," IEEE Trans. on Signal Processing, vol. 68, pp. 6198-6211, Oct. 2020.

See Also

fit_AR1_t

Examples

library(imputeFin)
data(ts_VAR_t)
fitted <- fit_VAR_t(Y = ts_VAR_t$Y, p = 2, parallel_max_cores = 2)

Impute missing values of time series based on a Gaussian AR(1) model

Description

Impute inner missing values (excluding leading and trailing ones) of time series by drawing samples from the conditional distribution of the missing values given the observed data based on a Gaussian AR(1) model as estimated with the function fit_AR1_Gaussian. Outliers can be detected and removed.

Usage

impute_AR1_Gaussian(
  y,
  n_samples = 1,
  random_walk = FALSE,
  zero_mean = FALSE,
  remove_outliers = FALSE,
  outlier_prob_th = 0.001,
  verbose = TRUE,
  return_estimates = FALSE,
  tol = 1e-10,
  maxiter = 100
)

Arguments

Value

By default (i.e., for n_samples = 1 and return_estimates = FALSE), the function will return an imputed time series of the same class and dimensions as the argument y with one new attribute recording the locations of missing values (the function plot_imputed will make use of such information to indicate the imputed values), as well as locations of outliers removed.

If n_samples > 1, the function will return a list consisting of n_sample imputed time series with names: y_imputed.1, y_imputed.2, etc.

If return_estimates = TRUE, in addition to the imputed time series y_imputed, the function will return the estimated model parameters:

Author(s)

Junyan Liu and Daniel P. Palomar

References

R. J. Little and D. B. Rubin, Statistical Analysis with Missing Data, 2nd ed. Hoboken, N.J.: John Wiley & Sons, 2002.

J. Liu, S. Kumar, and D. P. Palomar, "Parameter estimation of heavy-tailed AR model with missing data via stochastic EM," IEEE Trans. on Signal Processing, vol. 67, no. 8, pp. 2159-2172, 15 April, 2019.

See Also

plot_imputed, fit_AR1_Gaussian, impute_AR1_t

Examples

library(imputeFin)
data(ts_AR1_Gaussian) 
y_missing <- ts_AR1_Gaussian$y_missing
y_imputed <- impute_AR1_Gaussian(y_missing)
plot_imputed(y_imputed)

Impute missing values of time series based on a Student's t AR(1) model

Description

Impute inner missing values (excluding leading and trailing ones) of time series by drawing samples from the conditional distribution of the missing values given the observed data based on a Student's t AR(1) model as estimated with the function fit_AR1_t. Outliers can be detected and removed.

Usage

impute_AR1_t(
  y,
  n_samples = 1,
  random_walk = FALSE,
  zero_mean = FALSE,
  fast_and_heuristic = TRUE,
  remove_outliers = FALSE,
  outlier_prob_th = 0.001,
  verbose = TRUE,
  return_estimates = FALSE,
  tol = 1e-08,
  maxiter = 100,
  K = 30,
  n_burn = 100,
  n_thin = 50
)

Arguments

Value

By default (i.e., for n_samples = 1 and return_estimates = FALSE), the function will return an imputed time series of the same class and dimensions as the argument y with one new attribute recording the locations of missing values (the function plot_imputed will make use of such information to indicate the imputed values), as well as locations of outliers removed.

If n_samples > 1, the function will return a list consisting of n_sample imputed time series with names: y_imputed.1, y_imputed.2, etc.

If return_estimates = TRUE, in addition to the imputed time series y_imputed, the function will return the estimated model parameters:

Author(s)

Junyan Liu and Daniel P. Palomar

References

J. Liu, S. Kumar, and D. P. Palomar, "Parameter estimation of heavy-tailed AR model with missing data via stochastic EM," IEEE Trans. on Signal Processing, vol. 67, no. 8, pp. 2159-2172, 15 April, 2019.

See Also

plot_imputed, fit_AR1_t, impute_AR1_Gaussian

Examples

library(imputeFin)
data(ts_AR1_t) 
y_missing <- ts_AR1_t$y_missing
y_imputed <- impute_AR1_t(y_missing)
plot_imputed(y_imputed)

Impute missing values of an OHLC time series on a rolling window basis based on a Gaussian AR(1) model

Description

Impute inner missing values (excluding leading and trailing ones) of an OHLC time series on a rolling window basis. This is a wrapper of the functions impute_AR1_Gaussian and impute_rolling_AR1_Gaussian.

Usage

impute_OHLC(
  y_OHLC,
  rolling_window = 252,
  remove_outliers = FALSE,
  outlier_prob_th = 0.001,
  tol = 1e-10,
  maxiter = 100
)

Arguments

Value

Imputed OHLC prices.

Author(s)

Daniel P. Palomar

See Also

impute_AR1_Gaussian, impute_rolling_AR1_Gaussian

Impute missing values of time series on a rolling window basis based on a Gaussian AR(1) model

Description

Impute inner missing values (excluding leading and trailing ones) of time series on a rolling window basis. This is a wrapper of the function impute_AR1_Gaussian.

Usage

impute_rolling_AR1_Gaussian(
  y,
  rolling_window = 252,
  random_walk = FALSE,
  zero_mean = FALSE,
  remove_outliers = FALSE,
  outlier_prob_th = 0.001,
  tol = 1e-10,
  maxiter = 100
)

Arguments

Value

Same as impute_AR1_Gaussian for the case n_samples = 1 and return_estimates = FALSE.

Author(s)

Daniel P. Palomar

See Also

plot_imputed, impute_AR1_Gaussian

Examples

library(imputeFin)
data(ts_AR1_Gaussian) 
y_missing <- ts_AR1_Gaussian$y_missing
y_imputed <- impute_rolling_AR1_Gaussian(y_missing)
plot_imputed(y_imputed)

Plot imputed time series.

Description

Plot single imputed time series (as returned by functions impute_AR1_Gaussian and impute_AR1_t), highlighting the imputed values in a different color.

Usage

plot_imputed(
  y_imputed,
  column = 1,
  title = "Imputed time series",
  color_imputed = "red",
  type = c("ggplot2", "simple")
)

Arguments

Author(s)

Daniel P. Palomar

Examples

library(imputeFin)
data(ts_AR1_t) 
y_missing <- ts_AR1_t$y_missing
y_imputed <- impute_AR1_t(y_missing)
plot_imputed(y_missing, title = "Original time series with missing values")
plot_imputed(y_imputed)

Synthetic AR(1) Gaussian time series with missing values

Description

Synthetic AR(1) Gaussian time series with missing values for estimation and imputation testing purposes.

Usage

data(ts_AR1_Gaussian)

Format

List with the following elements:

y_missing

300 x 3 zoo object with three AR(1) Gaussian time series along the columns: the first column contains a time series with 10% consecutive missing values; the second column contains a time series with 10% missing values randomly distributed; and the third column contains the union of the previous missing values.

phi0

Value of phi0 used to generate the time series.

phi1

Value of phi1 used to generate the time series.

sigma2

Value of sigma2 used to generate the time series.

Synthetic AR(1) Student's t time series with missing values

Description

Synthetic AR(1) Student's t time series with missing values for estimation and imputation testing purposes.

Usage

data(ts_AR1_t)

Format

List with the following elements:

y_missing

300 x 3 zoo object with three AR(1) Student's t time series along the columns: the first column contains a time series with 10% consecutive missing values; the second column contains a time series with 10% missing values randomly distributed; and the third column contains the union of the previous missing values.

phi0

Value of phi0 used to generate the time series.

phi1

Value of phi1 used to generate the time series.

sigma2

Value of sigma2 used to generate the time series.

nu

Value of nu used to generate the time series.

Synthetic Student's t VAR data with missing values

Description

Synthetic Student's t VAR data with missing values for estimation and imputation testing purposes.

Usage

data(ts_VAR_t)

Format

List with the following elements:

Y

200 x 3 zoo object as a Student's t VAR time series.

phi0

True value of the constant vector in the VAR model.

Phii

True value of the coefficient matrix in the VAR model.

scatter

True value of the scatter matrix (of the noise distribution) in the VAR model.

nu

True value of the degrees of freedom (of the noise distribution) in the VAR model.