Package {exactGMH}

exactGMH: Exact/Permutation-Based Mantel and Generalized Mantel-Haenszel Tests for Differential Item Functioning

Description

Screens dichotomous and polytomous test items for Differential Item Functioning (DIF) using an extension of the Mantel (1963) / generalized Mantel-Haenszel statistic, with statistical significance computed via permutation rather than the conventional asymptotic chi-square approximation. This makes the test valid for small samples (n < 200 per group), a condition common in classroom-, program-, and institution-level assessment where existing exact-inference options (e.g., difR::difMH, stats::mantelhaen.test) are restricted to dichotomous items only.

Main functions

run_dif_screening

Screen all items in a response matrix for DIF (dichotomous and polytomous items may be mixed).

exact_mantel_dif

Run the exact/permutation Mantel test for a single item.

Author(s)

Maintainer: Tri Zahra Ningsih trizahra10019@gmail.com

Authors:

• Aman

• Ahmad Nasrulloh

• Hera Hastuti herahastuti@fis.unp.ac.id

• Suci Kurnia Putri

References

Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58(303), 690-700. doi:10.1080/01621459.1963.10500879

Hemerik, J., & Goeman, J. (2018). Exact testing with random permutations. TEST, 27(4), 811-825. doi:10.1007/s11749-017-0571-1

See Also

Useful links:

• https://github.com/exactGMH-project/exactGMH

• Report bugs at https://github.com/exactGMH-project/exactGMH/issues

Exact/permutation Mantel test for a single item

Description

Computes the Mantel (1963) / Mantel-Haenszel DIF statistic for a single dichotomous or polytomous item, with statistical significance obtained via permutation of group membership within score-matched strata. Following Hemerik and Goeman (2018), the observed statistic is included among the reference draws (the "+1" correction), which guarantees the resulting p-value is exact at the nominal level rather than merely an asymptotic approximation, even for a finite number of permutations.

Usage

exact_mantel_dif(item_score, group, total_score, n_perm = 5000, seed = 123)

Arguments

Value

A one-row data frame with columns:

item_type

"Dichotomous" or "Polytomous" (auto-detected).

z_statistic

Standardized Mantel statistic.

effect_size

Delta-MH (dichotomous) or standardized difference (polytomous).

effect_label

Label describing the effect size column.

chi_asymp

Asymptotic chi-square statistic (for comparison).

p_asymptotic

p-value from the conventional asymptotic chi-square approximation.

p_exact_perm

Exact permutation-based p-value (recommended).

classification

ETS-style DIF classification: "A (negligible)", "B (moderate)", or "C (large)".

n_permutations

Number of permutations used.

References

Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58(303), 690-700. doi:10.1080/01621459.1963.10500879

Hemerik, J., & Goeman, J. (2018). Exact testing with random permutations. TEST, 27(4), 811-825. doi:10.1007/s11749-017-0571-1

Examples

set.seed(1)
n <- 60
group <- rep(c("Reference", "Focal"), each = n / 2)
item <- rbinom(n, 1, 0.5)
total <- rowSums(replicate(5, rbinom(n, 1, 0.5)))
exact_mantel_dif(item, group, total, n_perm = 500)

Screen all items in a test for Differential Item Functioning

Description

Runs the exact/permutation Mantel test (see exact_mantel_dif) on every column of a response matrix. Dichotomous (0/1) and polytomous (0..C-1) items may be freely mixed within the same matrix; item type is auto-detected per column.

Usage

run_dif_screening(
  response_matrix,
  group,
  n_perm = 5000,
  seed = 123,
  p_adjust_method = "none"
)

Arguments

Value

A data frame with one row per item; see exact_mantel_dif for column descriptions. An additional item column identifies each item (from colnames(response_matrix) if available), and, when p_adjust_method != "none", a p_exact_adjusted column is added and used to update the classification column.

Examples

set.seed(42)
n <- 80
group <- rep(c("Reference", "Focal"), each = n / 2)
# 3 dichotomous items, 2 polytomous items (0-3)
resp <- data.frame(
  Item1 = rbinom(n, 1, 0.5),
  Item2 = rbinom(n, 1, 0.5),
  Item3 = rbinom(n, 1, 0.5),
  Item4 = sample(0:3, n, replace = TRUE),
  Item5 = sample(0:3, n, replace = TRUE)
)
run_dif_screening(resp, group, n_perm = 500, p_adjust_method = "BH")