Cramer - von Mises statistics

Description

Calculates the Cramer-von Mises test statistic

T(S_n)=\frac{1}{2q}\sum_{i=1}^{2q}\left(H^-_n(S_{n,i})-H^+_n(S_{n,i})\right)^2

where H^-_n(\cdot) and H^+_n(\cdot) are the empirical CDFs of the the sample of baseline covariates close to the cutoff from the left and right, respectively. See equation (12) in Canay and Kamat (2017).

Usage

CvM.stat(Sn)

Arguments

Value

Returns the numeric value of the Cramer - von Mises test statistic.

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Canay, I and Kamat V, (2018) Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design. The Review of Economic Studies, 85(3): 1577-1608

Regression Discontinuity Design Permutation test

Description

Calculates the empirical CDF of the sample of W conditional on Z being close to the cutoff from either the left or right. Given the induced order for the baseline covariates

W^{-}_{[q]}, W^{-}_{[q-1]},\dots\le W^{-}_{[1]}

or

W^{+}_{[1]}, W^{+}_{[2]},\dots, W^{+}_{[q]}

, this function will calculate either

H^-_n(t)=\frac{1}{q}\sum_{i=1}^q I\{W^{-}_{[i]}\le t\}

or

H^+_n(t)=\frac{1}{q}\sum_{i=1}^q I\{W^{+}_{[i]}\le t\}

depending on the argument of the function. See section 3 in Canay & Kamat (2017).

Usage

H.cdf(W, t)

Arguments

Value

Numeric. For a sample W=(w_1,\dots,w_n), returns the fraction of observations less or equal to t.

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Canay, I and Kamat V, (2018) Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design. The Review of Economic Studies, 85(3): 1577-1608

Martingale transformed Permutation Test: Multiple Testing procedures.

Description

This function applies the martingale transformed Permutation test (Chung and Olivares (2020)) to test whether there exists within-group treatment effect heterogeneity. The method jointly tests the null hypotheses that treatment effects are constant within mutually exclusive subgroups while allowing them to be different across subgroups. More formally, assume the mutually exclusive subgroups are formed from observed covariates, and are taken as given. Denote \mathcal{J} the total number of such subgroups. Let F_0^{j}(y) and F_1^{j}(y) be the CDFs of the control and treatment group for subgroup 1\le j\le \mathcal{J}. The null hypothesis of interest is given by the joint hypothesis

\mathbf{H}_{0}: F_1^{j}(y + \delta_{j}) = F_0^{j}(y)

for all mutually exclusive j\in\{1,\dots,\mathcal{J}\}, for some \delta_j. We are treating \mathbf{H}_0 as a multiple testing problem in which every individual hypothesis j\in\{1,\dots,\mathcal{J}\}, given by

H_{0,j}: F_1^{j}(y + \delta_{j}) = F_0^{j}(y)

for some \delta_j specifies whether the treatment effect is heterogeneous for a particular subgroup.

To achieve control of the family-wise error rate, the function considers several multiple testing procedures, such as Bonferroni, maxT and minP (Westfall and Young (1993)), and Holm (1979). For further details, see Chung and Olivares (2020).

Usage

PT.Khmaladze.MultTest(
  data,
  procedure = "maxT",
  alpha = 0.05,
  n.perm = 499,
  B = 499,
  na.action
)

Arguments

Value

An object of class "PT.Khmaladze.MultTest" is a list containing at least the following components:

Author(s)

Maurcio Olivares

References

Chung, E. and Olivares, M. (2021). Permutation Test for Heterogeneous Treatment Effects with a Nuisance Parameter. Forthcoming in Journal of Econometrics. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, pages 65-70. Westfall, P.H. and Young, S.S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment, Volume 279, John & Wiley Sons.

Examples

## Not run: 
subgroup1 <- list()
subgroup1$Y0 <- rnorm(11)
subgrpup1$Y1 <- rnorm(8,1,1) 
subgroup2 <- list()
subgroup2$Y0 <- rnorm(9)
subgroup2$Y1 <- rnorm(7,1,2)
data <- list(subgroup1,subgroup2)
res.minP <- PT.Khmaladze.MultTest(data,"minP",n.perm=100,B=100)
summary(res.minP)
adjusted.p.values <- res.minP$adj.pvalues
adjusted.p.values

## End(Not run)

Permutation Test for Heterogeneous Treatment Effects with a Nuisance Parameter

Description

A permutation test of the two-sample goodness-of-fit hypothesis in the presence of an estimated niusance parameter. The permutation test considered here is based on the Khmaladze transformation of the empirical process (Khmaladze (1981)), and adapted by Chung and Olivares (2020).

Usage

PT.Khmaladze.fit(y1, y0, alpha = 0.05, n.perm = 999)

Arguments

Value

An object of class "PT.Khmaladze.fit" containing at least the following components:

Author(s)

Maurcio Olivares

References

Khmaladze, E. (1981). Martingale Approach in the Theory of Goodness-of-fit Tests. Theory of Probability and its Application, 26: 240–257. Chung, E. and Olivares, M. (2021). Permutation Test for Heterogeneous Treatment Effects with a Nuisance Parameter. Forthcoming in Journal of Econometrics.

Examples

## Not run: 
Y0 <- rnorm(100, 1, 1)
# Treatment Group with constant shift equals to 1
Y1 <- Y0 + 1
Tx = sample(100) <= 0.5*(100)
# Observed Outcome 
Y = ifelse( Tx, Y1, Y0 )
dta <- data.frame(Y = Y, Z = as.numeric(Tx))
pt.GoF<-PT.Khmaladze.fit(dta$Y[dta$Z==1],dta$Y[dta$Z==0],n.perm = 49)
summary(pt.GoF)

## End(Not run)

Quantile-Based Permutation Test with an Estimated Nuisance Parameter

Description

A permutation test for testing whether the quantile treatment effects are constant across quantiles. The permutation test considered here is based on the Khmaladze transformation of the quantile process (Koenler and Xiao (2002)), and adapted by Chung and Olivares (2021).

Usage

PTQTE.Khmaladze.fit(
  Y,
  Z,
  taus = seq(0.1, 0.9, by = 0.05),
  alpha = 0.05,
  n.perm = 999
)

Arguments

Value

An object of class "PTQTE.Khmaladze" containing at least the following components:

Author(s)

Maurcio Olivares

References

Khmaladze, E. (1981). Martingale Approach in the Theory of Goodness-of-fit Tests. Theory of Probability and its Application, 26: 240–257. Koenker, R. and Xiao, Z. (2002) Inference on the Quantile Regression Process. Econometrica, 70(4): 1583-1612. Chung, E. and Olivares, M. (2021). Comment on "Can Variation in Subgroups' Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social Experiment."

Examples

## Not run: 
dta <- data.frame(Y=rnorm(100),Z=sample(c(0,1), 100, replace = TRUE))
pt.QTE<-PTQTE.Khmaladze.fit(dta$Y,dta$Z,taus=seq(.1,.9,by=0.05),alpha=0.05,n.perm = 499)
summary(pt.QTE)

## End(Not run)

Regression Discontinuity Design Permutation Test

Description

A permutation test for continuity of covariates in Sharp Regression Discontinuity Design as described in Canay and Kamat (2018).

Usage

RDperm(
  W,
  z,
  data,
  n.perm = 499,
  q_type = 10,
  cutoff = 0,
  test.statistic = "CvM"
)

Arguments

Value

The functions summary and plot are used to obtain and print a summary and plot of the estimated regression discontinuity. The object of class RDperm is a list containing the following components:

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Canay, I and Kamat V, (2018) Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design. The Review of Economic Studies, 85(3): 1577-1608

Examples

permtest<-RDperm(W=c("demshareprev"),z="difdemshare",data=lee2008)
summary(permtest)
## Not run: 
permtest<-RDperm(W=c("demshareprev","demwinprev"),z="difdemshare",data=lee2008)
summary(permtest)

## End(Not run)

Robust Permutation Test

Description

This function considers the k-sample problem of comparing general parameters, such as means, medians, or parameters that depend on the joint distribution using permutation tests. Under weak assumptions for comparing estimator, the permutation tests implemented here provide a general test procedure whereby the asymptotic validity of the permutation test holds while retaining the exact rejection probability \alpha in finite samples when the underlying distributions are identical. Here we will consider three test for the 2 sample case, but the function works for k-samples.

Difference of means: Here, the null hypothesis is of the form H_0: \mu(P)-\mu(Q)=0, and the corresponding test statistic is given by

T_{m,n}=\frac{N^{1/2}(\bar{X}_m-\bar{Y}_n)}{\sqrt{\frac{N}{m}\sigma^2_m(X_1,\dots,X_m)+ \frac{N}{n}\sigma^2_n(Y_1,\dots,Y_n)}}

where \bar{X}_m and \bar{Y}_n are the sample means from population P and population Q, respectively, and \sigma^2_m(X_1,\dots,X_m) is a consistent estimator of \sigma^2(P) when X_1,\dots,X_m are i.i.d. from P. Assume consistency also under Q.

Difference of medians: Let F and G be the CDFs corresponding to P and Q, and denote \theta(F) the median of F i.e. \theta(F)=\inf\{x:F(x)\ge1/2\}. Assume that F is continuously differentiable at \theta(P) with derivative F' (and the same with F replaced by G). Here, the null hypothesis is of the form H_0: \theta(P)-\theta(Q)=0, and the corresponding test statistic is given by

T_{m,n}=\frac{N^{1/2}\left(\theta(\hat{P}_m)-\theta(\hat{Q})\right)}{\hat{\upsilon}_{m,n}}

where \hat{\upsilon}_{m,n} is a consistent estimator of \upsilon(P,Q):

\upsilon(P,Q)=\frac{1}{\lambda}\frac{1}{4(F'(\theta))^2}+\frac{1}{1-\lambda}\frac{1}{4(G'(\theta))^2}

Choices of \hat{\upsilon}_{m,n} may include the kernel estimator of Devroye and Wagner (1980), the bootstrap estimator of Efron (1992), or the smoothed bootstrap Hall et al. (1989) to list a few. For further details, see Chung and Romano (2013). Current implementation uses the bootstrap estimator of Efron (1992)

Difference of variances: Here, the null hypothesis is of the form H_0: \sigma^2(P)-\sigma^2(Q)=0, and the corresponding test statistic is given by

T_{m,n}=\frac{N^{1/2}(\hat{\sigma}_m^2(X_1,\dots,X_,)-\hat{\sigma}_n^2(Y_1,\dots,Y_n))}{\sqrt{\frac{N}{m}(\hat{\mu}_{4,x}-\frac{(m-3)}{(m-1)}(\hat{\sigma}_m^2)^2)+\frac{N}{n}(\hat{\mu}_{4,y}-\frac{(n-3)}{(n-1)}(\hat{\sigma}_y^2)^2)}}

where \hat{\mu}_{4,m} the sample analog of E(X-\mu)^4 based on an i.i.d. sample X_1,\dots,X_m from P. Similarly for \hat{\mu}_{4,n}.

We could also have the case when the parameter of interest is a function of the joint distribution. The examples considered here are

Lehmann (1951) two-sample U statistics: Consider testing H_0: P=Q, or the more general hypothesis that P and Q only differ in location against the alternative that the Y's are more spread out than the X's. The null hypothesis is of the form

H_0: P(\vert Y-Y'\vert>\vert X-X'\vert)=1/2

.

Two-sample Wilcoxon statistic, where the null hypothesis is of the form

H_0: P(X\le Y)=1/2

.

Two-sample Wilcoxon statistic without continuity assumption. In this case, the null hypothesis is of the form

H_0: P(X\le Y)=P(Y\le X)

.

Hollander (1967) two-sample U statistics. The null hypothesis is of the form

H_0: P(X+X'<Y+Y')=1/2

.

Usage

RPT(
  formula,
  data,
  test = "means",
  n.perm = 499,
  na.action,
  wilcoxon.option = "continuity"
)

Arguments

Value

An object of class "RPT" is a list containing at least the following components:

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Chung, E. and Romano, J. P. (2013). Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2):484–507. Chung, E. and Romano, J. P. (2016). Asymptotically valid and exact permutation tests based on two-sample u-statistics. Journal of Statistical Planning and Inference, 168:97–105. Devroye, L. P. and Wagner, T. J. (1980). The strong uniform consistency of kernel density estimates. In Multivariate Analysis V: Proceedings of the fifth International Symposium on Multivariate Analysis, volume 5, pages 59–77. Efron, B. (1992). Bootstrap methods: another look at the jackknife. In Breakthroughs in statistics, pages 569–593. Springer. Hall, P., DiCiccio, T. J., and Romano, J. P. (1989). On smoothing and the bootstrap. The Annals of Statistics, pages 692–704. Hollander, M. (1967). Asymptotic efficiency of two nonparametric competitors of wilcoxon’s two sample test. Journal of the American Statistical Association, 62(319):939–949. Lehmann, E. L. (1951). Consistency and unbiasedness of certain nonparametric tests. The Annals of Mathematical Statistics, pages 165–179.

Examples

## Not run: 
male<-rnorm(50,1,1)
female<-rnorm(50,1,2)
dta<-data.frame(group=c(rep(1,50),rep(2,50)),outcome=c(male,female))
rpt.var<-RPT(dta$outcome~dta$group,test="variances")
summary(rpt.var)


## End(Not run)

General Construction of Permutation Tests: Group Actions

Description

Calculates the pre-specified actions on data. Consider data Z taking values in a sample space \Omega. Let \mathbf{G} be a finite group of transformations from \Omega onto itself, with M=\vert \mathbf{G}\vert. This function applies gZ as g varies in \bf{G}. If Z is a vector of sizeN and the actions g are permutations, M=N!. If the actions g are sign changes, then M=\{1,-1\}^{N}.

Usage

group.action(Z, M, type = "permutations")

Arguments

Value

Numeric. A matrix of size N\times M where N is the size of input Z and M is the number of actions to be performed on Z.

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Lehmann, Erich L. and Romano, Joseph P (2005) Testing statistical hypotheses.Springer Science & Business Media.

Data set used in Lee (2008)

Description

Randomized experiments from non-random selection in U.S. House elections

Format

A data frame with 6558 observations and two variables:

demsharenext

Democrat vote share election t+1

difdemshare

Running variable. Diff. democratic share

demshareprev

Democrat vote share t-1

demwinprev

Democrat win t-1

demofficeexp

Democrat political experience t

othofficeexp

Oppositions political experience t

demelectexp

Democrat electoral experience t

othelectexp

Oposition electoral experience t

Source

Mostly Harmless Econometrics Data Archive: https://economics.mit.edu/people/faculty/josh-angrist/mhe-data-archive

References

Lee, D. (2008) Randomized experiments from non-random selection in U.S. House elections, Journal of Econometrics, 142, 675-697

Plot RDperm

Description

Plots a histogram and empirical cdf

Usage

## S3 method for class 'RDperm'
plot(x, w, plot.class = "both", ...)

Arguments

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Canay, I and Kamat V, (2018) Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design. The Review of Economic Studies, 85(3): 1577-1608

Examples

## Not run: 
permtest<-RDperm(W=c("demshareprev","demwinprev"),z="difdemshare",data=lee2008)
plot(permtest,w="demshareprev")

## End(Not run)

Permutation Test for the two-sample goodness-of-fit problem under covariate-adaptive randomization

Description

A permutation test of the two-sample goodness-of-fit hypothesis when the randomization scheme is covariate-adaptive. The permutation test considered here is based on prepivoting the Kolmogorov-Smirnov test statistic following Beran (1987,1988), and adapted by Olivares (2020). Current version includes the following randomization schemes: simple randomization, Efron's biased-coin design, Wei's biased-coin design, and stratified block randomization. This implementation uses a Bayesian bootstrap approximation for prepivoting.

Usage

prepivot.ks.permtest(Y1, Y0, alpha, B, n.perm)

Arguments

Value

An object of class "prepivot.ks.permtest" containing at least the following components:

Author(s)

Maurcio Olivares

References

Beran, R. (1987). Prepivoting to reduce level error of confidence sets. Biometrika, 74(3): 457–468. Beran, R. (1988). Prepivoting test statistics: a bootstrap view of asymptotic refinements. Journal of the American Statistical Association, 83(403):687–697. Olivares, M. (2020). Asymptotically Robust Permutation Test under Covariate-Adaptive Randomization. Working Paper.

Examples

## Not run: 
Y0 <- rnorm(100, 1, 1)
Y1 <- rbeta(100,2,2)
Tx = sample(100) <= 0.5*(100)
# Observed Outcome 
Y = ifelse( Tx, Y1, Y0 )
dta <- data.frame(Y = Y, A = as.numeric(Tx))
pKS.GoF<-prepivot.ks.permtest(dta$Y[dta$A==1],dta$Y[dta$A==0],alpha=0.05,B=1000,n.perm = 999)
summary(pKS.GoF)

## End(Not run)

General Construction of Randomization Tests

Description

Calculates the randomization test. Further discussion can be found in chapter 15 of Lehmann and Romano (2005, p 633). Consider data X taking values in a sample space \Omega. Let \mathbf{G} be a finite group of transformations from \Omega onto itself, with M=\vert \mathbf{G}\vert. Let T(X) be a real-valued test statistic such that large values provide evidence against the null hypothesis. Denote by

T^{(1)}(X)\le T^{(2)}(X)\le\dots\le T^{(M)}(X)

the ordered values of \{T(gX)\,:\,g\in\mathbf{G}\}. Let k=M-\lfloor M\alpha\rfloor and define M^{+}(x) and M^{0}(x) be the number of values T^{(j)}(X), j=1,\dots,M, which are greater than T^{(k)}(X) and equal to T^{(k)}(X) respectively. Set

a(X)=\frac{\alpha M-M^{+}(X)}{M^{0}(X)}~.

The randomization test is given by

\phi(X)=1\{T(x)> T^{(k)}(X)\}+a(X)\times 1\{T(X)= T^{(k)}(X)\}~.

Usage

randomization.test(Tn, Tng, alpha = 0.05)

Arguments

Value

Numeric. A vector containing \phi(X)\in\{0,1\} and T^{(k)}(X). The test rejects the null hypothesis if \phi(X)=1, and does not reject otherwise.

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

References

Lehmann, Erich L. and Romano, Joseph P (2005) Testing statistical hypotheses.Springer Science & Business Media.

Summarizing Permutation Test for Within-grpup Treatment Effect Heterogeneity in the presence of an Estimated Nuisance Parameter

Description

summary method for class "PT.Khmaladze.fit"

Usage

## S3 method for class 'PT.Khmaladze.MultTest'
summary(object, ..., digits = max(3, getOption("digits") - 3))

Arguments

Value

summary.PT.Khmaladze.MultTest returns an object of class "summary.PT.Khmaladze.MultTest" which has the following components

Author(s)

Maurcio Olivares

Summarizing Permutation Test for Heterogeneous Treatment Effects with Estimated Nuisance Parameter

Description

summary method for class "PT.Khmaladze.fit"

Usage

## S3 method for class 'PT.Khmaladze.fit'
summary(object, ..., digits = max(3, getOption("digits") - 3))

Arguments

Value

summary.PT.Khmaladze.fit returns an object of class "summary.PT.Khmaladze.fit" which has the following components

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

Summarizing Quantile-Based Permutation Test with an Estimated Nuisance Parameter

Description

summary method for class "PTQTE.Khmaladze.fit"

Usage

## S3 method for class 'PTQTE.Khmaladze.fit'
summary(object, ..., digits = max(3, getOption("digits") - 3))

Arguments

Value

PTQTE.Khmaladze.fit returns an object of class "PTQTE.Khmaladze.fit" which has the following components

Author(s)

Maurcio Olivares

Summarizing Regression Discontinuity Design Permutation Test

Description

summary method for class "RDPerm"

Usage

## S3 method for class 'RDperm'
summary(object, digits = max(3, getOption("digits") - 3), ...)

Arguments

Value

summary.RDperm returns an object of class "summary.RDperm" which has the following components

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

Summarizing Robust Permutation Test

Description

summary method for class "RPT"

Usage

## S3 method for class 'RPT'
summary(object, ..., digits = max(3, getOption("digits") - 3))

Arguments

Value

summary.RPT returns an object of class "summary.RPT" which has the following components

Author(s)

Maurcio Olivares

Ignacio Sarmiento Barbieri

Summarizing Two-sample Goodness-of-fit Permutation Test under Covariate-adaptive Randomization

Description

summary method for class "prepivot.ks.permtest"

Usage

## S3 method for class 'prepivot.ks.permtest'
summary(object, ..., digits = max(3, getOption("digits") - 3))

Arguments

Value

summary.prepivot.ks.permtest returns an object of class "summary.prepivot.ks.permtest" which has the following components

Author(s)

Maurcio Olivares