Variable Clustering with Multiple Latent Components Clustering algorithm

Description

Package varclust performs clustering of variables, according to a probabilistic model, which assumes that each cluster lies in a low dimensional subspace. Segmentation of variables, number of clusters and their dimensions are selected based on the appropriate implementation of the Bayesian Information Criterion.

Details

The best candidate models are identified by the specific implementation of K-means algorithm, in which cluster centers are represented by some number of orthogonal factors(principal components of the variables within a cluster) and similarity between a given variable and a cluster center depends on residuals from a linear model fit. Based on the Bayesian Information Criterion (BIC), sums of squares of residuals are appropriately scaled, which allows to avoid an over-excessive attraction by clusters with larger dimensions. To reduce the chance that the local minimum of modified BIC (mBIC) is obtained instead of the global one, for every fixed number of clusters in a given range K-means algorithm is run large number of times, with different random initializations of cluster centers.

The main function of package varclust is mlcc.bic which allows clustering variables in a data with unknown number of clusters. Variable partition is computed with k-means based algorithm. Number of clusters and their dimensions are estimated using mBIC and PESEL respectively. If the number of clusters is known one might use function mlcc.reps, which takes number of clusters as a parameter. For mlcc.reps one might specify as well some initial segmentation for k-means algorithm. This can be useful if user has some a priori knowledge about clustering.

We provide also two functions to simulate datasets with described structure. The function data.simulation generates the data so that the subspaces are indepentend and data.simulation.factors generates the data where some factores are shared between the subspaces.

We also provide function measures of quality of clustering. misclassification computes misclassification rate between two partitions. This performance measure is extensively used in image segmentation. The other measure is implemented as integration function.

Version: 0.9.4

Author(s)

Piotr Sobczyk, Stanislaw Wilczynski, Julie Josse, Malgorzata Bogdan

Maintainer: Piotr Sobczyk pj.sobczyk@gmail.com

Examples

sim.data <- data.simulation(n = 50, SNR = 1, K = 3, numb.vars = 50, max.dim = 3)
mlcc.bic(sim.data$X, numb.clusters = 1:5, numb.runs = 20, numb.cores = 1, verbose = TRUE)
mlcc.reps(sim.data$X, numb.clusters = 3, numb.runs = 20, numb.cores = 1)

Calculates principal components for every cluster

Description

For given segmentation this function estimates dimensionality of each cluster (or chooses fixed dimensionality) and for each cluster calculates the number of principal components equal to the this dimensionality

Usage

calculate.pcas(X, segmentation, number.clusters, max.subspace.dim,
  estimate.dimensions)

Arguments

Value

A subset of principal components for every cluster.

Choses a subspace for a variable

Description

Selects a subspace closest to a given variable. To select the subspace, the method considers (for every subspace) a subset of its principal components and tries to fit a linear model with the variable as the response. Then the method chooses the subspace for which the value of BIC was the highest.

Usage

choose.cluster.BIC(variable, pcas, number.clusters,
  show.warnings = FALSE)

Arguments

Value

index Number of most similar subspace to variable.

mBIC for subspace clustering

Description

Computes the value of modified Bayesian Information Criterion (mBIC) for given data set partition and clusters' dimensionalities. In each cluster we assume that variables are spanned by few factors. Considering maximum likelihood we get that those factors are in fact principal components. Additionally, it uses by default an informative prior distribution on models.

Usage

cluster.pca.BIC(X, segmentation, dims, numb.clusters, max.dim,
  flat.prior = FALSE)

Arguments

Value

Value of mBIC

Simulates subspace clustering data

Description

Generates data for simulation with a low-rank subspace structure: variables are clustered and each cluster has a low-rank representation. Factors than span subspaces are not shared between clusters.

Usage

data.simulation(n = 100, SNR = 1, K = 10, numb.vars = 30,
  max.dim = 2, min.dim = 1, equal.dims = TRUE)

Arguments

Value

A list consisting of:

Examples

sim.data <- data.simulation()
sim.data2 <- data.simulation(n = 30, SNR = 2, K = 5, numb.vars = 20, 
                             max.dim = 3, equal.dims = FALSE)

Simulates subspace clustering data with shared factors

Description

Generating data for simulation with a low-rank subspace structure: variables are clustered and each cluster has a low-rank representation. Factors that span subspaces are shared between clusters.

Usage

data.simulation.factors(n = 100, SNR = 1, K = 10, numb.vars = 30,
  numb.factors = 10, min.dim = 1, max.dim = 2, equal.dims = TRUE,
  separation.parameter = 0.1)

Arguments

Value

A list consisting of:

Examples

sim.data <- data.simulation.factors()
sim.data2 <- data.simulation.factors(n = 30, SNR = 2, K = 5, numb.vars = 20,
             numb.factors = 10, max.dim = 3, equal.dims = FALSE, separation.parameter = 0.2)

Computes integration and acontamination of the clustering

Description

Integartion and acontamination are measures of the quality of a clustering with a reference to a true partition. Let X = (x_1, \ldots x_p) be the data set, A be a partition into clusters A_1, \ldots A_n (true partition) and B be a partition into clusters B_1, \ldots, B_m. Then for cluster A_j integration is eqaul to:

Int(A_j) = \frac{max_{k = 1, \ldots, m} \# \{ i \in \{ 1, \ldots p \}: x_i \in A_j \wedge x_i \in B_k \} }{\# A_j}

The B_k for which the value is maximized is called the integrating cluster of A_j. Then the integration for the whole clustering equals is Int(A,B) = \frac{1}{n} \sum_{j=1}^n Int(A_j) .The acontamination is defined by:

Acont(A_j) = \frac{ \# \{ i \in \{ 1, \ldots p \}: x_i \in A_j \wedge x_i \in B_k \} }{\# B_k}

where B_k is the integrating cluster for A_j. Then the acontamination for the whole dataset is Acont(A,B) = \frac{1}{n} \sum_{j=1}^n Acont(A_j)

Usage

integration(group, true_group)

Arguments

Value

An array containing values of integration and acontamination.

References

M. Sołtys. Metody analizy skupień. Master’s thesis, Wrocław University of Technology, 2010

Examples

sim.data <- data.simulation(n = 20, SNR = 1, K = 2, numb.vars = 50, max.dim = 2)
true_segmentation <- rep(1:2, each=50)
mlcc.fit <- mlcc.reps(sim.data$X, numb.clusters = 2, max.dim = 2, numb.cores=1)
integration(mlcc.fit$segmentation, true_segmentation)

Computes misclassification rate

Description

Missclasification is a commonly used performance measure in subspace clustering. It allows to compare two partitions with the same number of clusters.

Usage

misclassification(group, true_group, M, K)

Arguments

Details

As getting exact value of misclassification requires checking all permutations and is therefore intrackable even for modest number of clusters, a heuristic approach is proposed. It is assumed that there are K classes of maximum M elements. Additional requirement is that classes labels are from range [1, K].

Value

Misclassification rate.

References

R. Vidal. Subspace clustering. Signal Processing Magazine, IEEE, 28(2):52-68,2011

Examples

sim.data <- data.simulation(n = 100, SNR = 1, K = 5, numb.vars = 30, max.dim = 2)
mlcc.fit <- mlcc.reps(sim.data$X, numb.clusters = 5, numb.runs = 20, max.dim = 2, numb.cores=1)
misclassification(mlcc.fit$segmentation,sim.data$s, 30, 5)


#one can use this function not only for clusters
partition1 <- sample(10, 300, replace = TRUE)
partition2 <- sample(10, 300, replace = TRUE)
misclassification(partition1, partition1, max(table(partition1)), 10)
misclassification(partition1, partition2, max(table(partition2)), 10)

Multiple Latent Components Clustering - Subspace clustering with automatic estimation of number of clusters and their dimension

Description

This function is an implementation of Multiple Latent Components Clustering (MLCC) algorithm which clusteres quantitative variables into a number, chosen using mBIC, of groups. For each considered number of clusters in numb.clusters mlcc.reps function is called. It invokes K-means based algorithm (mlcc.kmeans) finding local minimum of mBIC, which is run a given number of times (numb.runs) with different initializations. The best partition is choosen with mBIC (see mlcc.reps function).

Usage

mlcc.bic(X, numb.clusters = 1:10, numb.runs = 30, stop.criterion = 1,
  max.iter = 30, max.dim = 4, scale = TRUE, numb.cores = NULL,
  greedy = TRUE, estimate.dimensions = TRUE, verbose = FALSE,
  flat.prior = FALSE, show.warnings = FALSE)

Arguments

Value

An object of class mlcc.fit consisting of

Examples

sim.data <- data.simulation(n = 50, SNR = 1, K = 3, numb.vars = 50, max.dim = 3)
mlcc.res <- mlcc.bic(sim.data$X, numb.clusters = 1:5, numb.runs = 20, numb.cores = 1, verbose=TRUE)
show.clusters(sim.data$X, mlcc.res$segmentation)

Multiple Latent Components Clustering - kmeans algorithm

Description

Performs k-means based subspace clustering. Center of each cluster is some number of principal components. This number can be fixed or estimated by PESEL. Similarity measure between variable and a cluster is calculated using BIC.

Usage

mlcc.kmeans(X, number.clusters = 2, stop.criterion = 1,
  max.iter = 30, max.subspace.dim = 4, initial.segmentation = NULL,
  estimate.dimensions = TRUE, show.warnings = FALSE)

Arguments

Value

A list consisting of:

References

Bayesian dimensionality reduction with PCA using penalized semi-integrated likelihood, Piotr Sobczyk, Malgorzata Bogdan, Julie Josse

Examples

sim.data <- data.simulation(n = 50, SNR = 1, K = 5, numb.vars = 50, max.dim = 3)
mlcc.res <- mlcc.kmeans(sim.data$X, number.clusters = 5, max.iter = 20, max.subspace.dim = 3)
show.clusters(sim.data$X, mlcc.res$segmentation)

Multiple Latent Components Clustering - Subspace clustering assuming that the number of clusters is known

Description

For a fixed number of cluster function returns the best partition and basis for each subspace.

Usage

mlcc.reps(X, numb.clusters = 2, numb.runs = 30, stop.criterion = 1,
  max.iter = 30, initial.segmentations = NULL, max.dim = 4,
  scale = TRUE, numb.cores = NULL, estimate.dimensions = TRUE,
  flat.prior = FALSE, show.warnings = FALSE)

Arguments

Details

In more detail, an algorithm mlcc.kmeans is run a numb.runs of times with random or custom initializations. The best partition is selected according to the BIC.

Value

A list consisting of

Examples

sim.data <- data.simulation(n = 50, SNR = 1, K = 5, numb.vars = 50, max.dim = 3)
mlcc.res <- mlcc.reps(sim.data$X, numb.clusters = 5, numb.runs = 20, max.dim = 4, numb.cores = 1)
show.clusters(sim.data$X, mlcc.res$segmentation)

Plot mlcc.fit class object

Description

Plot mlcc.fit class object

Usage

## S3 method for class 'mlcc.fit'
plot(x, ...)

Arguments

Print mlcc.fit class object

Description

Print mlcc.fit class object

Usage

## S3 method for class 'mlcc.fit'
print(x, ...)

Arguments

Print mlcc.reps.fit class object

Description

Print mlcc.reps.fit class object

Usage

## S3 method for class 'mlcc.reps.fit'
print(x, ...)

Arguments

Print clusters obtained from MLCC

Description

Print clusters obtained from MLCC

Usage

show.clusters(data, segmentation)

Arguments