Type: Package
Title: Multiple Hypotheses Testing for Discrete Data
Version: 1.0.1
Author: Yalin Zhu, Wenge Guo
Maintainer: Yalin Zhu <yalin.zhu@outlook.com>
BugReports: https://github.com/allenzhuaz/MHTdiscrete/issues
URL: https://allen.shinyapps.io/MTPs/
Description: A comprehensive tool for almost all existing multiple testing methods for discrete data. The package also provides some novel multiple testing procedures controlling FWER/FDR for discrete data. Given discrete p-values and their domains, the [method].p.adjust function returns adjusted p-values, which can be used to compare with the nominal significant level alpha and make decisions. For users' convenience, the functions also provide the output option for printing decision rules.
License: GPL-2 | GPL-3 [expanded from: GPL (≥ 2)]
LazyData: TRUE
NeedsCompilation: no
Packaged: 2018-12-15 20:39:44 UTC; yalinzhu
Repository: CRAN
Date/Publication: 2018-12-15 21:00:02 UTC

MHTdiscrete: A package for Multiple Hypotheses Testing for Discrete Data.

Description

The MHTdiscrete package provides two categories of important functions for discrete data mutliple hypothese testing:

FWER controlling procedures

Single-step: MBonf.p.adjust, MixBonf.p.adjust, Tarone.p.adjust.

Step-down: MHolm.p.adjust, TH.p.adjust.

Step-up: MHoch.p.adjust, Roth.p.adjust .

FDR controlling procedures

Step-down: MBL.p.adjust.

Step-up: MBH.p.adjust, GTBH.p.adjust, MBY.p.adjust, GTBY.p.adjust.

Author(s)

Yalin Zhu

References

Zhu, Y., & Guo, W. (2017). Familywise error rate controlling procedures for discrete data arXiv preprint arXiv:1711.08147.

Tarone, R. E. (1990). A modified Bonferroni method for discrete data. Biometrics, 46: 515-522.

Hommel, G., & Krummenauer, F. (1998). Improvements and modifications of Tarone's multiple test procedure for discrete data. Biometrics, 54: 673-681.

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6: 65-70.

Roth, A. J. (1999). Multiple comparison procedures for discrete test statistics. Journal of statistical planning and inference, 82: 101-117.

Gilbert, P. B. (2005). A modified false discovery rate multiple-comparisons procedure for discrete data, applied to human immunodeficiency virus genetics. Journal of the Royal Statistical Society: Series C (Applied Statistics), 54: 143-158.

Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B, 57: 289-300.

Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29: 1165-1188.

Benjamini, Y., and Liu, W. (1999). A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. Journal of Statistical Planning and Inference, 82: 163-170.

The adjusted p-values for Gilbert-Tarone-BH step-up FDR controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

GTBH.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Gilbert, P. B. (2005). A modified false discovery rate multiple-comparisons procedure for discrete data, applied to human immunodeficiency virus genetics. Journal of the Royal Statistical Society: Series C (Applied Statistics), 54: 143-158.

Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B, 57: 289-300.

See Also

GTBY.p.adjust, MBH.p.adjust, MBY.p.adjust

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
GTBH.p.adjust(p,p.set)

The adjusted p-values for Gilbert-Tarone-BY step-up FDR controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

GTBY.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Gilbert, P. B. (2005). A modified false discovery rate multiple-comparisons procedure for discrete data, applied to human immunodeficiency virus genetics. Journal of the Royal Statistical Society: Series C (Applied Statistics), 54: 143-158.

Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29: 1165-1188.

See Also

GTBH.p.adjust, MBH.p.adjust, MBY.p.adjust

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
GTBY.p.adjust(p,p.set)

The adjusted p-values for Modified Benjamini-Hochberg (BH) step-up FDR controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

MBH.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B, 57: 289-300.

See Also

MBY.p.adjust, MBL.p.adjust

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
MBH.p.adjust(p,p.set)

The adjusted p-values for Modified Benjamini-Liu (BL) step-down FDR controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

MBL.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Note

The MBL procedure for discrete data controls FDR under the specific dependence assumption where the joint distribution of statistics from true nulls are independent of the joint distribution of statistics from false nulls.

Author(s)

Yalin Zhu

References

Benjamini, Y., and Liu, W. (1999). A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. Journal of Statistical Planning and Inference, 82: 163-170.

See Also

MBH.p.adjust, MBY.p.adjust

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
MBL.p.adjust(p,p.set)

The adjusted p-values for Modified Benjamini-Yekutieli (BY) step-up FDR controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

MBY.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29: 1165-1188.

See Also

MBH.p.adjust, MBL.p.adjust

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
MBY.p.adjust(p,p.set)

The adjusted p-values for Modified Bonferroni single-step FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values

Usage

MBonf.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSE, or a list including original p-values, adjusted p-values and decision rules if make.decision = TRUE.

Note

The attainable p-value refers to the element of domain set of p-value for the corresponding hypothesis. For continuous test statistics, the p-value under true null are uniform distributed in (0,1), thus the p-values are attainable everywhere between 0 and 1. But for discrete test statistics, the p-value can only take finite values bewtween 0 and 1, that is the attainable p-values for discrete case are finite and countable, so we can assign them in a finite list p.set.

Author(s)

Yalin Zhu

References

Zhu, Y., & Guo, W. (2017). Familywise error rate controlling procedures for discrete data arXiv preprint arXiv:1711.08147.

See Also

Tarone.p.adjust, MixBonf.p.adjust, p.adjust.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
MBonf.p.adjust(p,p.set)
## Compare with the traditional Bonferroni adjustment
p.adjust(p,method = "bonferroni")
## Compare with the Tarone adjustment
Tarone.p.adjust(p,p.set)

The adjusted p-values for Modified Hochberg step-up FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

MHoch.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis..

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Zhu, Y., & Guo, W. (2017). Familywise error rate controlling procedures for discrete data arXiv preprint arXiv:1711.08147.

Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75: 800-803.

See Also

Roth.p.adjust, p.adjust.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
MHoch.p.adjust(p,p.set)
## Compare with the traditional Hochberg adjustment
p.adjust(p,method = "hochberg")
## Compare with the Roth adjustment
Roth.p.adjust(p,p.set)

The adjusted p-values for Modified Holm step-down FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

MHolm.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Zhu, Y., & Guo, W. (2017). Familywise error rate controlling procedures for discrete data arXiv preprint arXiv:1711.08147.

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6: 65-70.

See Also

TH.p.adjust, p.adjust.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
MHolm.p.adjust(p,p.set)
## Compare with the traditional Holm adjustment
p.adjust(p,method = "holm")
## Compare with the Tarone-Holm adjustment
TH.p.adjust(p,p.set)

The adjusted p-values for Mixed Bonferroni single-step FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and the attaianble p-values for the discrete test statistics.

Usage

MixBonf.p.adjust(pc, pd, pd.set, alpha, make.decision)

Arguments

pc

numeric vector of the available p-values (possibly with NAs) for the continuous test statistics. Any other R is coerced by as.numeric. Same as in p.adjust.

pd

numeric vector of the available p-values (possibly with NAs) for the discrete test statistics. Any other R is coerced by as.numeric. Same as in p.adjust.

pd.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis for discrete data.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSE, or a list including original p-values, adjusted p-values and decision rules if make.decision = TRUE.

Note

The arguments include three parts, the available p-values need to be reorganized in advance. Gather all available p-values for continuous data as pc, and all available p-values for discrete data as pd. The attainable p-value refers to the element of domain set of p-value for the corresponding hypothesis for discrete test statistics, the p-value can only take finite values bewtween 0 and 1, that is, the attainable p-values for discrete case are finite and countable, so we can assign them in a finite list pd.set. The function returns the adjusted p-values with the first part for continuous data of the same length as pc, and second part for discrete data of the same length as pd

Author(s)

Yalin Zhu

References

Zhu, Y., & Guo, W. (2017). Familywise error rate controlling procedures for discrete data arXiv preprint arXiv:1711.08147.

See Also

Tarone.p.adjust, MBonf.p.adjust, p.adjust.

Examples

pd <- c(pbinom(1,8,0.5),pbinom(1,5,0.75)); pc <- c(0.04, 0.1)
pd.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75))
MixBonf.p.adjust(pc,pd,pd.set)
## Compare with the traditional Bonferroni adjustment
p.adjust(c(pc,pd),method = "bonferroni")

The adjusted p-values for Roth's step-up FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

Roth.p.adjust(p, p.set, digits,  alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis..

digits

minimal number of significant digits for the adjusted p-values, the default value is 4, see print.default.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Roth, A. J. (1999). Multiple comparison procedures for discrete test statistics. Journal of statistical planning and inference, 82: 101-117.

See Also

MHoch.p.adjust, p.adjust.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
Roth.p.adjust(p,p.set,digits=5)

The number of rejected hypotheses for Roth's step-up FWER controlling procedure.

Description

The function for calculating the number of rejected hypotheses (rejection region) based on original available p-values, all attaianble p-values and the given significant level.

Usage

Roth.rej(p,p.set,alpha)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis..

alpha

the given significant level for Roth's procedure, the default value is 0.05.

Value

An integer value of the number of rejected hypotheses.

Author(s)

Yalin Zhu

References

Roth, A. J. (1999). Multiple comparison procedures for discrete test statistics. Journal of statistical planning and inference, 82: 101-117.

Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75: 800-803.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
Roth.rej(p,p.set,0.05)

The adjusted p-values for Sidak single-step FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values.

Usage

Sidak.p.adjust(p, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSE, or a list including original p-values, adjusted p-values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

See Also

p.adjust.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
Sidak.p.adjust(p)

The adjusted p-values for Tarone-Holm step-down FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

TH.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Hommel, G., & Krummenauer, F. (1998). Improvements and modifications of Tarone's multiple test procedure for discrete data. Biometrics, 54: 673-681.

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6: 65-70.

See Also

MHolm.p.adjust, p.adjust.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
TH.p.adjust(p,p.set)

The adjusted p-values for Tarone's single-step FWER controlling procedure.

Description

The function for calculating the adjusted p-values based on original available p-values and all attaianble p-values.

Usage

Tarone.p.adjust(p, p.set, alpha, make.decision)

Arguments

p

numeric vector of p-values (possibly with NAs). Any other R is coerced by as.numeric. Same as in p.adjust.

p.set

a list of numeric vectors, where each vector is the vector of all attainable p-values containing the available p-value for the corresponding hypothesis.

alpha

significant level used to compare with adjusted p-values to make decisions, the default value is 0.05.

make.decision

logical; if TRUE, then the output include the decision rules compared adjusted p-values with significant level \alpha

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSE, or a list including original p-values, adjusted p-values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Tarone, R. E. (1990). A modified Bonferroni method for discrete data. Biometrics, 46: 515-522.

See Also

MBonf.p.adjust, MixBonf.p.adjust, p.adjust.

Examples

p <- c(pbinom(1,8,0.5),pbinom(1,5,0.75),pbinom(1,6,0.6))
p.set <-list(pbinom(0:8,8,0.5),pbinom(0:5,5,0.75),pbinom(0:6,6,0.6))
Tarone.p.adjust(p,p.set)

Calculating p-values for discrete data

Description

The function for calculating the original available p-values and all attaianble p-values for the corresponding hypothesis.

Usage

getPval(raw.data, test.type, alternative)

Arguments

raw.data

original data set with count number for treatment group and study group. The data set type could be matrix or data.frame.

test.type

there are two discrete test available now, must be one of "FET" for Fisher's Exact Test and "BET" for Binomial Exact Test.

alternative

indicates the alternative hypothesis and must be one of "two.sided", "greater" or "less".

Value

A numeric vector of the adjusted p-values (of the same length as p).

Author(s)

Yalin Zhu

References

Zhu, Y., & Guo, W. (2017). Familywise error rate controlling procedures for discrete data arXiv preprint arXiv:1711.08147.

Clopper, C. J. & Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26: 404-413.

Fisher, R. A. (1922). On the Interpretation of \chi^2 from Contingency Tables, and the Calculation of P. Journal of the Royal Statistical Society, 85: 87-94.

Examples

 ## Using Fisher's Exact Test to get the avaiable and attainablep-values
 # import raw data set as data.frame type
 df <-  data.frame(X1=c(4, 2, 2, 13, 6, 8, 4, 0, 1), N1 = rep(148, 9),
 	X2 = c(0, 0, 1, 3, 2, 1, 2, 2, 2), N2 = rep(132, 9))
 # obtain the avaiable p-values and attainable p-values using two-sided Fisher's Exact Test
 getPval(raw.data=df, test.type = "FET",alternative = "two.sided")
 # store the avaiable p-values
p <- getPval(raw.data=df, test.type = "FET",alternative = "two.sided")[[1]]; p
 # store the attainable p-values
p.set <- getPval(raw.data=df, test.type = "FET",alternative = "two.sided")[[2]]; p.set