Type:	Package
Title:	Artificial Intelligence for Education
Version:	1.0.2
Description:	In social and educational settings, the use of Artificial Intelligence (AI) is a challenging task. Relevant data is often only available in handwritten forms, or the use of data is restricted by privacy policies. This often leads to small data sets. Furthermore, in the educational and social sciences, data is often unbalanced in terms of frequencies. To support educators as well as educational and social researchers in using the potentials of AI for their work, this package provides a unified interface for neural nets in 'PyTorch' to deal with natural language problems. In addition, the package ships with a shiny app, providing a graphical user interface. This allows the usage of AI for people without skills in writing python/R scripts. The tools integrate existing mathematical and statistical methods for dealing with small data sets via pseudo-labeling (e.g. Cascante-Bonilla et al. (2020) <doi:10.48550/arXiv.2001.06001>) and imbalanced data via the creation of synthetic cases (e.g. Bunkhumpornpat et al. (2012) <doi:10.1007/s10489-011-0287-y>). Performance evaluation of AI is connected to measures from content analysis which educational and social researchers are generally more familiar with (e.g. Berding & Pargmann (2022) <doi:10.30819/5581>, Gwet (2014) <ISBN:978-0-9708062-8-4>, Krippendorff (2019) <doi:10.4135/9781071878781>). Estimation of energy consumption and CO2 emissions during model training is done with the 'python' library 'codecarbon'. Finally, all objects created with this package allow to share trained AI models with other people.
License:	GPL-3
URL:	https://fberding.github.io/aifeducation/
BugReports:	https://github.com/cran/aifeducation/issues
Depends:	R (≥ 3.5.0)
Imports:	doParallel, foreach, iotarelr(≥ 0.1.5), irrCAC, methods, Rcpp (≥ 1.0.10), reshape2, reticulate (≥ 1.34.0), rlang, smotefamily, stringi, utils
Suggests:	bslib, DT, fs, future, ggplot2, knitr, promises, readtext, readxl, rmarkdown, shiny(≥ 1.9.0), shinyFiles, shinyWidgets, sortable, testthat (≥ 3.0.0)
LinkingTo:	Rcpp, RcppArmadillo
VignetteBuilder:	knitr
Config/testthat/edition:	3
Encoding:	UTF-8
LazyData:	true
RoxygenNote:	7.3.2
SystemRequirements:	PyTorch (see vignette "Get started")
NeedsCompilation:	yes
Packaged:	2025-02-05 12:39:46 UTC; WissMit
Author:	Berding Florian [aut, cre], Tykhonova Yuliia [aut], Pargmann Julia [ctb], Leube Anna [ctb], Riebenbauer Elisabeth [ctb], Rebmann Karin [ctb], Slopinski Andreas [ctb]
Maintainer:	Berding Florian <florian.berding@uni-hamburg.de>
Repository:	CRAN
Date/Publication:	2025-02-05 13:00:02 UTC

Base R6 class for creation and definition of .AIFE*Transformer-like classes

Description

This base class is used to create and define .AIFE*Transformer-like classes. It serves as a skeleton for a future concrete transformer and cannot be used to create an object of itself (an attempt to call new-method will produce an error).

See p.1 Base Transformer Class in Transformers for Developers for details.

Create

The create-method is a basic algorithm that is used to create a new transformer, but cannot be called directly.

Train

The train-method is a basic algorithm that is used to train and tune the transformer but cannot be called directly.

Concrete transformer implementation

There are already implemented concrete (child) transformers (e.g. BERT, DeBERTa-V2, etc.), to implement a new one see p.4 Implement A Custom Transformer in Transformers for Developers

Public fields

Methods

Public methods

• .AIFEBaseTransformer$new()

• .AIFEBaseTransformer$set_title()

• .AIFEBaseTransformer$set_model_param()

• .AIFEBaseTransformer$set_model_temp()

• .AIFEBaseTransformer$set_SFC_check_max_pos_emb()

• .AIFEBaseTransformer$set_SFC_create_tokenizer_draft()

• .AIFEBaseTransformer$set_SFC_calculate_vocab()

• .AIFEBaseTransformer$set_SFC_save_tokenizer_draft()

• .AIFEBaseTransformer$set_SFC_create_final_tokenizer()

• .AIFEBaseTransformer$set_SFC_create_transformer_model()

• .AIFEBaseTransformer$set_required_SFC()

• .AIFEBaseTransformer$set_SFT_load_existing_model()

• .AIFEBaseTransformer$set_SFT_cuda_empty_cache()

• .AIFEBaseTransformer$set_SFT_create_data_collator()

• .AIFEBaseTransformer$create()

• .AIFEBaseTransformer$train()

• .AIFEBaseTransformer$clone()

Method new()

An object of this class cannot be created. Thus, method's call will produce an error.

Usage

.AIFEBaseTransformer$new()

Returns

This method returns an error.

Method set_title()

Setter for the title. Sets a new value for the title private attribute.

Usage

.AIFEBaseTransformer$set_title(title)

Arguments

Returns

This method returns nothing.

Method set_model_param()

Setter for the parameters. Adds a new parameter and its value to the params list.

Usage

.AIFEBaseTransformer$set_model_param(param_name, param_value)

Arguments

Returns

This method returns nothing.

Method set_model_temp()

Setter for the temporary model's parameters. Adds a new temporary parameter and its value to the temp list.

Usage

.AIFEBaseTransformer$set_model_temp(temp_name, temp_value)

Arguments

Returns

This method returns nothing.

Method set_SFC_check_max_pos_emb()

Setter for the check_max_pos_emb element of the private steps_for_creation list. Sets a new fun function as the check_max_pos_emb step.

Usage

.AIFEBaseTransformer$set_SFC_check_max_pos_emb(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_create_tokenizer_draft()

Setter for the create_tokenizer_draft element of the private steps_for_creation list. Sets a new fun function as the create_tokenizer_draft step.

Usage

.AIFEBaseTransformer$set_SFC_create_tokenizer_draft(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_calculate_vocab()

Setter for the calculate_vocab element of the private steps_for_creation list. Sets a new fun function as the calculate_vocab step.

Usage

.AIFEBaseTransformer$set_SFC_calculate_vocab(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_save_tokenizer_draft()

Setter for the save_tokenizer_draft element of the private steps_for_creation list. Sets a new fun function as the save_tokenizer_draft step.

Usage

.AIFEBaseTransformer$set_SFC_save_tokenizer_draft(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_create_final_tokenizer()

Setter for the create_final_tokenizer element of the private steps_for_creation list. Sets a new fun function as the create_final_tokenizer step.

Usage

.AIFEBaseTransformer$set_SFC_create_final_tokenizer(fun)

Arguments

Returns

This method returns nothing.

Method set_SFC_create_transformer_model()

Setter for the create_transformer_model element of the private steps_for_creation list. Sets a new fun function as the create_transformer_model step.

Usage

.AIFEBaseTransformer$set_SFC_create_transformer_model(fun)

Arguments

Returns

This method returns nothing.

Method set_required_SFC()

Setter for all required elements of the private steps_for_creation list. Executes setters for all required creation steps.

Usage

.AIFEBaseTransformer$set_required_SFC(required_SFC)

Arguments

Returns

This method returns nothing.

Method set_SFT_load_existing_model()

Setter for the load_existing_model element of the private steps_for_training list. Sets a new fun function as the load_existing_model step.

Usage

.AIFEBaseTransformer$set_SFT_load_existing_model(fun)

Arguments

Returns

This method returns nothing.

Method set_SFT_cuda_empty_cache()

Setter for the cuda_empty_cache element of the private steps_for_training list. Sets a new fun function as the cuda_empty_cache step.

Usage

.AIFEBaseTransformer$set_SFT_cuda_empty_cache(fun)

Arguments

Returns

This method returns nothing.

Method set_SFT_create_data_collator()

Setter for the create_data_collator element of the private steps_for_training list. Sets a new fun function as the create_data_collator step. Use this method to make a custom data collator for a transformer.

Usage

.AIFEBaseTransformer$set_SFT_create_data_collator(fun)

Arguments

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the child-transformer architecture and a vocabulary using the python libraries transformers and tokenizers.

This method adds the following parameters to the temp list:

• log_file

• raw_text_dataset

• pt_safe_save

• value_top

• total_top

• message_top

This method uses the following parameters from the temp list:

• log_file

• raw_text_dataset

• tokenizer

Usage

.AIFEBaseTransformer$create(
  ml_framework,
  model_dir,
  text_dataset,
  vocab_size,
  max_position_embeddings,
  hidden_size,
  num_attention_heads,
  intermediate_size,
  hidden_act,
  hidden_dropout_prob,
  attention_probs_dropout_prob,
  sustain_track,
  sustain_iso_code,
  sustain_region,
  sustain_interval,
  trace,
  pytorch_safetensors,
  log_dir,
  log_write_interval
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on BERT architecture with the help of the python libraries transformers, datasets, and tokenizers.

This method adds the following parameters to the temp list:

• log_file

• loss_file

• from_pt

• from_tf

• load_safe

• raw_text_dataset

• pt_safe_save

• value_top

• total_top

• message_top

This method uses the following parameters from the temp list:

• log_file

• raw_text_dataset

• tokenized_dataset

• tokenizer

Usage

.AIFEBaseTransformer$train(
  ml_framework,
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask,
  whole_word,
  val_size,
  n_epoch,
  batch_size,
  chunk_size,
  full_sequences_only,
  min_seq_len,
  learning_rate,
  n_workers,
  multi_process,
  sustain_track,
  sustain_iso_code,
  sustain_region,
  sustain_interval,
  trace,
  keras_trace,
  pytorch_trace,
  pytorch_safetensors,
  log_dir,
  log_write_interval
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEBaseTransformer$clone(deep = FALSE)

Arguments

References

Hugging Face transformers documantation:

• BERT

• DeBERTa

• Funnel

• Longformer

• RoBERTa

• MPNet

See Also

Other Transformers for developers: .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Child R6 class for creation and training of BERT transformers

Description

This class has the following methods:

• create: creates a new transformer based on BERT.

• train: trains and fine-tunes a BERT model.

Create

New models can be created using the .AIFEBertTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFEBertTransformer$train.

Pre-Trained models that can be fine-tuned using this method are available at https://huggingface.co/.

The model is trained using dynamic masking, as opposed to the original paper, which used static masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEBertTransformer

Methods

Public methods

• .AIFEBertTransformer$new()

• .AIFEBertTransformer$create()

• .AIFEBertTransformer$train()

• .AIFEBertTransformer$clone()

Method new()

Creates a new transformer based on BERT and sets the title.

Usage

.AIFEBertTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the BERT base architecture and a vocabulary based on WordPiece by using the python libraries transformers and tokenizers.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• num_hidden_layer

Usage

.AIFEBertTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on BERT architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFEBertTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.003,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEBertTransformer$clone(deep = FALSE)

Arguments

Note

This model uses a WordPiece tokenizer like BERT and can be trained with whole word masking. The transformer library may display a warning, which can be ignored.

References

Devlin, J., Chang, M.‑W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In J. Burstein, C. Doran, & T. Solorio (Eds.), Proceedings of the 2019 Conference of the North (pp. 4171–4186). Association for Computational Linguistics. doi:10.18653/v1/N19-1423

Hugging Face documentation

• https://huggingface.co/docs/transformers/model_doc/bert

• https://huggingface.co/docs/transformers/model_doc/bert#transformers.BertForMaskedLM

• https://huggingface.co/docs/transformers/model_doc/bert#transformers.TFBertForMaskedLM

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Child R6 class for creation and training of DeBERTa-V2 transformers

Description

This class has the following methods:

• create: creates a new transformer based on DeBERTa-V2.

• train: trains and fine-tunes a DeBERTa-V2 model.

Create

New models can be created using the .AIFEDebertaTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFEDebertaTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of this model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEDebertaTransformer

Methods

Public methods

• .AIFEDebertaTransformer$new()

• .AIFEDebertaTransformer$create()

• .AIFEDebertaTransformer$train()

• .AIFEDebertaTransformer$clone()

Method new()

Creates a new transformer based on DeBERTa-V2 and sets the title.

Usage

.AIFEDebertaTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the DeBERTa-V2 base architecture and a vocabulary based on the SentencePiece tokenizer using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• num_hidden_layer

Usage

.AIFEDebertaTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 128100,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 1536,
  num_hidden_layer = 24,
  num_attention_heads = 24,
  intermediate_size = 6144,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on DeBERTa-V2 architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFEDebertaTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.03,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEDebertaTransformer$clone(deep = FALSE)

Arguments

Note

For this model a WordPiece tokenizer is created. The standard implementation of DeBERTa version 2 from HuggingFace uses a SentencePiece tokenizer. Thus, please use AutoTokenizer from the transformers library to work with this model.

References

He, P., Liu, X., Gao, J. & Chen, W. (2020). DeBERTa: Decoding-enhanced BERT with Disentangled Attention. doi:10.48550/arXiv.2006.03654

Hugging Face documentatio

• https://huggingface.co/docs/transformers/model_doc/deberta-v2

• https://huggingface.co/docs/transformers/model_doc/deberta-v2#transformers.DebertaV2ForMaskedLM

• https://huggingface.co/docs/transformers/model_doc/deberta-v2#transformers.TFDebertaV2ForMaskedLM

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Child R6 class for creation and training of Funnel transformers

Description

This class has the following methods:

• create: creates a new transformer based on Funnel.

• train: trains and fine-tunes a Funnel model.

Create

New models can be created using the .AIFEFunnelTransformer$create method.

Model is created with separete_cls = TRUE, truncate_seq = TRUE, and pool_q_only = TRUE.

Train

To train the model, pass the directory of the model to the method .AIFEFunnelTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of the model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEFunnelTransformer

Methods

Public methods

• .AIFEFunnelTransformer$new()

• .AIFEFunnelTransformer$create()

• .AIFEFunnelTransformer$train()

• .AIFEFunnelTransformer$clone()

Method new()

Creates a new transformer based on Funnel and sets the title.

Usage

.AIFEFunnelTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the Funnel transformer base architecture and a vocabulary based on WordPiece using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• target_hidden_size

• block_sizes

• num_decoder_layers

• pooling_type

• activation_dropout

Usage

.AIFEFunnelTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 768,
  target_hidden_size = 64,
  block_sizes = c(4, 4, 4),
  num_attention_heads = 12,
  intermediate_size = 3072,
  num_decoder_layers = 2,
  pooling_type = "mean",
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  activation_dropout = 0,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on Funnel Transformer architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFEFunnelTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.003,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEFunnelTransformer$clone(deep = FALSE)

Arguments

Note

The model uses a configuration with truncate_seq = TRUE to avoid implementation problems with tensorflow.

This model uses a WordPiece tokenizer like BERT and can be trained with whole word masking. The transformer library may display a warning, which can be ignored.

References

Dai, Z., Lai, G., Yang, Y. & Le, Q. V. (2020). Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing. doi:10.48550/arXiv.2006.03236

Hugging Face documentation

• https://huggingface.co/docs/transformers/model_doc/funnel#funnel-transformer

• https://huggingface.co/docs/transformers/model_doc/funnel#transformers.FunnelModel

• https://huggingface.co/docs/transformers/model_doc/funnel#transformers.TFFunnelModel

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Child R6 class for creation and training of Longformer transformers

Description

This class has the following methods:

• create: creates a new transformer based on Longformer.

• train: trains and fine-tunes a Longformer model.

Create

New models can be created using the .AIFELongformerTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFELongformerTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of this model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFELongformerTransformer

Methods

Public methods

• .AIFELongformerTransformer$new()

• .AIFELongformerTransformer$create()

• .AIFELongformerTransformer$train()

• .AIFELongformerTransformer$clone()

Method new()

Creates a new transformer based on Longformer and sets the title.

Usage

.AIFELongformerTransformer$new()

Returns

This method returns nothing

Method create()

This method creates a transformer configuration based on the Longformer base architecture and a vocabulary based on ⁠Byte-Pair Encoding⁠ (BPE) tokenizer using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• add_prefix_space

• trim_offsets

• num_hidden_layer

• attention_window

Usage

.AIFELongformerTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  add_prefix_space = FALSE,
  trim_offsets = TRUE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  attention_window = 512,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on Longformer Transformer architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFELongformerTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.03,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFELongformerTransformer$clone(deep = FALSE)

Arguments

References

Beltagy, I., Peters, M. E., & Cohan, A. (2020). Longformer: The Long-Document Transformer. doi:10.48550/arXiv.2004.05150

Hugging Face Documentation

• https://huggingface.co/docs/transformers/model_doc/longformer

• https://huggingface.co/docs/transformers/model_doc/longformer#transformers.LongformerModel

• https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerModel

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer, .AIFETrObj

Child R6 class for creation and training of MPNet transformers

Description

This class has the following methods:

• create: creates a new transformer based on MPNet.

• train: trains and fine-tunes a MPNet model.

Create

New models can be created using the .AIFEMpnetTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFEMpnetTransformer$train.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFEMpnetTransformer

Public fields

Methods

Public methods

• .AIFEMpnetTransformer$new()

• .AIFEMpnetTransformer$create()

• .AIFEMpnetTransformer$train()

• .AIFEMpnetTransformer$clone()

Method new()

Creates a new transformer based on MPNet and sets the title.

Usage

.AIFEMpnetTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the MPNet base architecture.

This method adds the following 'dependent' parameters to the base class's inherited params list:

• vocab_do_lower_case

• num_hidden_layer

Usage

.AIFEMpnetTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  vocab_do_lower_case = FALSE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on MPNet architecture with the help of the python libraries transformers, datasets, and tokenizers.

This method adds the following 'dependent' parameter to the base class's inherited params list:

• p_perm

Usage

.AIFEMpnetTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  p_perm = 0.15,
  whole_word = TRUE,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.003,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = FALSE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFEMpnetTransformer$clone(deep = FALSE)

Arguments

Note

Using this class with tensorflow is not supported. Supported framework is pytorch.

References

Song,K., Tan, X., Qin, T., Lu, J. & Liu, T.-Y. (2020). MPNet: Masked and Permuted Pre-training for Language Understanding. doi:10.48550/arXiv.2004.09297

Hugging Face documentation

• https://huggingface.co/docs/transformers/model_doc/mpnet

• https://huggingface.co/docs/transformers/model_doc/mpnet#transformers.MPNetForMaskedLM

• https://huggingface.co/docs/transformers/model_doc/mpnet#transformers.TFMPNetForMaskedLM

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFERobertaTransformer, .AIFETrObj

Child R6 class for creation and training of RoBERTa transformers

Description

This class has the following methods:

• create: creates a new transformer based on RoBERTa.

• train: trains and fine-tunes a RoBERTa model.

Create

New models can be created using the .AIFERobertaTransformer$create method.

Train

To train the model, pass the directory of the model to the method .AIFERobertaTransformer$train.

Pre-Trained models which can be fine-tuned with this function are available at https://huggingface.co/.

Training of this model makes use of dynamic masking.

Super class

aifeducation::.AIFEBaseTransformer -> .AIFERobertaTransformer

Methods

Public methods

• .AIFERobertaTransformer$new()

• .AIFERobertaTransformer$create()

• .AIFERobertaTransformer$train()

• .AIFERobertaTransformer$clone()

Method new()

Creates a new transformer based on RoBERTa and sets the title.

Usage

.AIFERobertaTransformer$new()

Returns

This method returns nothing.

Method create()

This method creates a transformer configuration based on the RoBERTa base architecture and a vocabulary based on ⁠Byte-Pair Encoding⁠ (BPE) tokenizer using the python transformers and tokenizers libraries.

This method adds the following 'dependent' parameters to the base class' inherited params list:

• add_prefix_space

• trim_offsets

• num_hidden_layer

Usage

.AIFERobertaTransformer$create(
  ml_framework = "pytorch",
  model_dir,
  text_dataset,
  vocab_size = 30522,
  add_prefix_space = FALSE,
  trim_offsets = TRUE,
  max_position_embeddings = 512,
  hidden_size = 768,
  num_hidden_layer = 12,
  num_attention_heads = 12,
  intermediate_size = 3072,
  hidden_act = "gelu",
  hidden_dropout_prob = 0.1,
  attention_probs_dropout_prob = 0.1,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead, it saves the configuration and vocabulary of the new model to disk.

Method train()

This method can be used to train or fine-tune a transformer based on RoBERTa Transformer architecture with the help of the python libraries transformers, datasets, and tokenizers.

Usage

.AIFERobertaTransformer$train(
  ml_framework = "pytorch",
  output_dir,
  model_dir_path,
  text_dataset,
  p_mask = 0.15,
  val_size = 0.1,
  n_epoch = 1,
  batch_size = 12,
  chunk_size = 250,
  full_sequences_only = FALSE,
  min_seq_len = 50,
  learning_rate = 0.03,
  n_workers = 1,
  multi_process = FALSE,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  trace = TRUE,
  keras_trace = 1,
  pytorch_trace = 1,
  pytorch_safetensors = TRUE,
  log_dir = NULL,
  log_write_interval = 2
)

Arguments

Returns

This method does not return an object. Instead the trained or fine-tuned model is saved to disk.

Method clone()

The objects of this class are cloneable with this method.

Usage

.AIFERobertaTransformer$clone(deep = FALSE)

Arguments

References

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. doi:10.48550/arXiv.1907.11692

Hugging Face Documentation

• https://huggingface.co/docs/transformers/model_doc/roberta

• https://huggingface.co/docs/transformers/model_doc/roberta#transformers.RobertaModel

• https://huggingface.co/docs/transformers/model_doc/roberta#transformers.TFRobertaModel

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFETrObj

Transformer objects

Description

This list contains transformer objects. Elements of the list are used in the public make of the AIFETransformerMaker R6 class. This list is not designed to be used directly.

It has the following elements: bert, roberta, deberta_v2, funnel, longformer, mpnet

Usage

.AIFETrObj

Format

An object of class list of length 6.

See Also

Other Transformers for developers: .AIFEBaseTransformer, .AIFEBertTransformer, .AIFEDebertaTransformer, .AIFEFunnelTransformer, .AIFELongformerTransformer, .AIFEMpnetTransformer, .AIFERobertaTransformer

Base class for models using neural nets

Description

Abstract class for all models that do not rely on the python library 'transformers'.

Value

Objects of this containing fields and methods used in several other classes in 'ai for education'. This class is not designed for a direct application and should only be used by developers.

Public fields

Methods

Public methods

• AIFEBaseModel$get_model_info()

• AIFEBaseModel$get_text_embedding_model()

• AIFEBaseModel$set_publication_info()

• AIFEBaseModel$get_publication_info()

• AIFEBaseModel$set_model_license()

• AIFEBaseModel$get_model_license()

• AIFEBaseModel$set_documentation_license()

• AIFEBaseModel$get_documentation_license()

• AIFEBaseModel$set_model_description()

• AIFEBaseModel$get_model_description()

• AIFEBaseModel$save()

• AIFEBaseModel$load()

• AIFEBaseModel$get_package_versions()

• AIFEBaseModel$get_sustainability_data()

• AIFEBaseModel$get_ml_framework()

• AIFEBaseModel$get_text_embedding_model_name()

• AIFEBaseModel$check_embedding_model()

• AIFEBaseModel$count_parameter()

• AIFEBaseModel$is_configured()

• AIFEBaseModel$get_private()

• AIFEBaseModel$get_all_fields()

• AIFEBaseModel$clone()

Method get_model_info()

Method for requesting the model information.

Usage

AIFEBaseModel$get_model_info()

Returns

list of all relevant model information.

Method get_text_embedding_model()

Method for requesting the text embedding model information.

Usage

AIFEBaseModel$get_text_embedding_model()

Returns

list of all relevant model information on the text embedding model underlying the model.

Method set_publication_info()

Method for setting publication information of the model.

Usage

AIFEBaseModel$set_publication_info(authors, citation, url = NULL)

Arguments

Returns

Function does not return a value. It is used for setting the private members for publication information.

Method get_publication_info()

Method for requesting the bibliographic information of the model.

Usage

AIFEBaseModel$get_publication_info()

Returns

list with all saved bibliographic information.

Method set_model_license()

Method for setting the license of the model.

Usage

AIFEBaseModel$set_model_license(license = "CC BY")

Arguments

Returns

Function does not return a value. It is used for setting the private member for the software license of the model.

Method get_model_license()

Method for getting the license of the model.

Usage

AIFEBaseModel$get_model_license()

Arguments

Returns

string representing the license for the model.

Method set_documentation_license()

Method for setting the license of the model's documentation.

Usage

AIFEBaseModel$set_documentation_license(license = "CC BY")

Arguments

Returns

Function does not return a value. It is used for setting the private member for the documentation license of the model.

Method get_documentation_license()

Method for getting the license of the model's documentation.

Usage

AIFEBaseModel$get_documentation_license()

Arguments

Returns

Returns the license as a string.

Method set_model_description()

Method for setting a description of the model.

Usage

AIFEBaseModel$set_model_description(
  eng = NULL,
  native = NULL,
  abstract_eng = NULL,
  abstract_native = NULL,
  keywords_eng = NULL,
  keywords_native = NULL
)

Arguments

Returns

Function does not return a value. It is used for setting the private members for the description of the model.

Method get_model_description()

Method for requesting the model description.

Usage

AIFEBaseModel$get_model_description()

Returns

list with the description of the classifier in English and the native language.

Method save()

Method for saving a model.

Usage

AIFEBaseModel$save(dir_path, folder_name)

Arguments

Returns

Function does not return a value. It saves the model to disk.

Method load()

Method for importing a model.

Usage

AIFEBaseModel$load(dir_path)

Arguments

Returns

Function does not return a value. It is used to load the weights of a model.

Method get_package_versions()

Method for requesting a summary of the R and python packages' versions used for creating the model.

Usage

AIFEBaseModel$get_package_versions()

Returns

Returns a list containing the versions of the relevant R and python packages.

Method get_sustainability_data()

Method for requesting a summary of tracked energy consumption during training and an estimate of the resulting CO2 equivalents in kg.

Usage

AIFEBaseModel$get_sustainability_data()

Returns

Returns a list containing the tracked energy consumption, CO2 equivalents in kg, information on the tracker used, and technical information on the training infrastructure.

Method get_ml_framework()

Method for requesting the machine learning framework used for the model.

Usage

AIFEBaseModel$get_ml_framework()

Returns

Returns a string describing the machine learning framework used for the classifier.

Method get_text_embedding_model_name()

Method for requesting the name (unique id) of the underlying text embedding model.

Usage

AIFEBaseModel$get_text_embedding_model_name()

Returns

Returns a string describing name of the text embedding model.

Method check_embedding_model()

Method for checking if the provided text embeddings are created with the same TextEmbeddingModel as the model.

Usage

AIFEBaseModel$check_embedding_model(text_embeddings)

Arguments

Returns

TRUE if the underlying TextEmbeddingModel are the same. FALSE if the models differ.

Method count_parameter()

Method for counting the trainable parameters of a model.

Usage

AIFEBaseModel$count_parameter()

Returns

Returns the number of trainable parameters of the model.

Method is_configured()

Method for checking if the model was successfully configured. An object can only be used if this value is TRUE.

Usage

AIFEBaseModel$is_configured()

Returns

bool TRUE if the model is fully configured. FALSE if not.

Method get_private()

Method for requesting all private fields and methods. Used for loading and updating an object.

Usage

AIFEBaseModel$get_private()

Returns

Returns a list with all private fields and methods.

Method get_all_fields()

Return all fields.

Usage

AIFEBaseModel$get_all_fields()

Returns

Method returns a list containing all public and private fields of the object.

Method clone()

The objects of this class are cloneable with this method.

Usage

AIFEBaseModel$clone(deep = FALSE)

Arguments

Transformer types

Description

This list contains transformer types. Elements of the list can be used in the public make of the AIFETransformerMaker R6 class as input parameter type.

It has the following elements:

• bert = 'bert'

• roberta = 'roberta'

• deberta_v2 = 'deberta_v2'

• funnel = 'funnel'

• longformer = 'longformer'

• mpnet = 'mpnet'

  Elements can be used like AIFETrType$bert, AIFETrType$deberta_v2, AIFETrType$funnel, etc.

Usage

AIFETrType

Format

An object of class list of length 6.

See Also

Other Transformer: AIFETransformerMaker, aife_transformer_maker

R6 class for transformer creation

Description

This class was developed to make the creation of transformers easier for users. Pass the transformer's type to the make method and get desired transformer. Now run the create or/and train methods of the new transformer.

The already created aife_transformer_maker object of this class can be used.

See p.3 Transformer Maker in Transformers for Developers for details.

See .AIFEBaseTransformer class for details.

Methods

Public methods

• AIFETransformerMaker$make()

• AIFETransformerMaker$clone()

Method make()

Creates a new transformer with the passed type.

Usage

AIFETransformerMaker$make(type)

Arguments

Returns

If success - a new transformer, otherwise - an error (passed type is invalid).

Method clone()

The objects of this class are cloneable with this method.

Usage

AIFETransformerMaker$clone(deep = FALSE)

Arguments

See Also

Other Transformer: AIFETrType, aife_transformer_maker

Examples

# Create transformer maker
tr_maker <- AIFETransformerMaker$new()

# Use 'make' method of the 'tr_maker' object
# Pass string with the type of transformers
# Allowed types are "bert", "deberta_v2", "funnel", etc. See aifeducation::AIFETrType list
my_bert <- tr_maker$make("bert")

# Or use elements of the 'aifeducation::AIFETrType' list
my_longformer <- tr_maker$make(AIFETrType$longformer)

# Run 'create' or 'train' methods of the transformer in order to create a
# new transformer or train the newly created one, respectively
# my_bert$create(...)
# my_bert$train(...)

# my_longformer$create(...)
# my_longformer$train(...)

Data manager for classification tasks

Description

Abstract class for managing the data and samples during training a classifier. DataManagerClassifier is used with TEClassifierRegular and TEClassifierProtoNet.

Value

Objects of this class are used for ensuring the correct data management for training different types of classifiers. Objects of this class are also used for data augmentation by creating synthetic cases with different techniques.

Public fields

Methods

Public methods

• DataManagerClassifier$new()

• DataManagerClassifier$get_config()

• DataManagerClassifier$get_labeled_data()

• DataManagerClassifier$get_unlabeled_data()

• DataManagerClassifier$get_samples()

• DataManagerClassifier$set_state()

• DataManagerClassifier$get_n_folds()

• DataManagerClassifier$get_n_classes()

• DataManagerClassifier$get_statistics()

• DataManagerClassifier$get_dataset()

• DataManagerClassifier$get_val_dataset()

• DataManagerClassifier$get_test_dataset()

• DataManagerClassifier$create_synthetic()

• DataManagerClassifier$add_replace_pseudo_data()

• DataManagerClassifier$clone()

Method new()

Creating a new instance of this class.

Usage

DataManagerClassifier$new(
  data_embeddings,
  data_targets,
  folds = 5,
  val_size = 0.25,
  class_levels,
  one_hot_encoding = TRUE,
  add_matrix_map = TRUE,
  sc_methods = "dbsmote",
  sc_min_k = 1,
  sc_max_k = 10,
  trace = TRUE,
  n_cores = auto_n_cores()
)

Arguments

Returns

Method returns an initialized object of class DataManagerClassifier.

Method get_config()

Method for requesting the configuration of the DataManagerClassifier.

Usage

DataManagerClassifier$get_config()

Returns

Returns a list storing the configuration of the DataManagerClassifier.

Method get_labeled_data()

Method for requesting the complete labeled data set.

Usage

DataManagerClassifier$get_labeled_data()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing all cases with labels.

Method get_unlabeled_data()

Method for requesting the complete unlabeled data set.

Usage

DataManagerClassifier$get_unlabeled_data()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing all cases without labels.

Method get_samples()

Method for requesting the assignments to train, validation, and test data sets for every fold and the final training.

Usage

DataManagerClassifier$get_samples()

Returns

Returns a list storing the assignments to a train, validation, and test data set for every fold. In the case of the sample for the final training the test data set is always empty (NULL).

Method set_state()

Method for setting the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$set_state(iteration, step = NULL)

Arguments

Returns

Method does not return anything. It is used for setting the internal state of the DataManager.

Method get_n_folds()

Method for requesting the number of folds the DataManagerClassifier can use with the current data.

Usage

DataManagerClassifier$get_n_folds()

Returns

Returns the number of folds the DataManagerClassifier uses.

Method get_n_classes()

Method for requesting the number of classes.

Usage

DataManagerClassifier$get_n_classes()

Returns

Returns the number classes.

Method get_statistics()

Method for requesting descriptive sample statistics.

Usage

DataManagerClassifier$get_statistics()

Returns

Returns a table describing the absolute frequencies of the labeled and unlabeled data. The rows contain the length of the sequences while the columns contain the labels.

Method get_dataset()

Method for requesting a data set for training depending in the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$get_dataset(
  inc_labeled = TRUE,
  inc_unlabeled = FALSE,
  inc_synthetic = FALSE,
  inc_pseudo_data = FALSE
)

Arguments

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing the requested kind of data along with all requested transformations for training. Please note that this method returns a data sets that is designed for training only. The corresponding validation data set is requested with get_val_dataset and the corresponding test data set with get_test_dataset.

Method get_val_dataset()

Method for requesting a data set for validation depending in the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$get_val_dataset()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing the requested kind of data along with all requested transformations for validation. The corresponding data set for training can be requested with get_dataset and the corresponding data set for testing with get_test_dataset.

Method get_test_dataset()

Method for requesting a data set for testing depending in the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$get_test_dataset()

Returns

Returns an object of class datasets.arrow_dataset.Dataset containing the requested kind of data along with all requested transformations for validation. The corresponding data set for training can be requested with get_dataset and the corresponding data set for validation with get_val_dataset.

Method create_synthetic()

Method for generating synthetic data used during training. The process uses all labeled data belonging to the current state of the DataManagerClassifier.

Usage

DataManagerClassifier$create_synthetic(trace = TRUE, inc_pseudo_data = FALSE)

Arguments

Returns

This method does nothing return. It generates a new data set for synthetic cases which are stored as an object of class datasets.arrow_dataset.Dataset in the field datasets$data_labeled_synthetic. Please note that a call of this method will override an existing data set in the corresponding field.

Method add_replace_pseudo_data()

Method for adding data with pseudo labels generated by a classifier

Usage

DataManagerClassifier$add_replace_pseudo_data(inputs, labels)

Arguments

Returns

This method does nothing return. It generates a new data set for synthetic cases which are stored as an object of class datasets.arrow_dataset.Dataset in the field datasets$data_labeled_pseudo. Please note that a call of this method will override an existing data set in the corresponding field.

Method clone()

The objects of this class are cloneable with this method.

Usage

DataManagerClassifier$clone(deep = FALSE)

Arguments

See Also

Other Data Management: EmbeddedText, LargeDataSetForText, LargeDataSetForTextEmbeddings

Embedded text

Description

Object of class R6 which stores the text embeddings generated by an object of class TextEmbeddingModel. The text embeddings are stored within memory/RAM. In the case of a high number of documents the data may not fit into memory/RAM. Thus, please use this object only for a small sample of texts. In general, it is recommended to use an object of class LargeDataSetForTextEmbeddings which can deal with any number of texts.

Value

Returns an object of class EmbeddedText. These objects are used for storing and managing the text embeddings created with objects of class TextEmbeddingModel. Objects of class EmbeddedText serve as input for objects of class TEClassifierRegular, TEClassifierProtoNet, and TEFeatureExtractor. The main aim of this class is to provide a structured link between embedding models and classifiers. Since objects of this class save information on the text embedding model that created the text embedding it ensures that only embedding generated with same embedding model are combined. Furthermore, the stored information allows objects to check if embeddings of the correct text embedding model are used for training and predicting.

Public fields

Methods

Public methods

• EmbeddedText$configure()

• EmbeddedText$save()

• EmbeddedText$is_configured()

• EmbeddedText$load_from_disk()

• EmbeddedText$get_model_info()

• EmbeddedText$get_model_label()

• EmbeddedText$get_times()

• EmbeddedText$get_features()

• EmbeddedText$get_original_features()

• EmbeddedText$is_compressed()

• EmbeddedText$add_feature_extractor_info()

• EmbeddedText$get_feature_extractor_info()

• EmbeddedText$convert_to_LargeDataSetForTextEmbeddings()

• EmbeddedText$n_rows()

• EmbeddedText$get_all_fields()

• EmbeddedText$clone()

Method configure()

Creates a new object representing text embeddings.

Usage

EmbeddedText$configure(
  model_name = NA,
  model_label = NA,
  model_date = NA,
  model_method = NA,
  model_version = NA,
  model_language = NA,
  param_seq_length = NA,
  param_chunks = NULL,
  param_features = NULL,
  param_overlap = NULL,
  param_emb_layer_min = NULL,
  param_emb_layer_max = NULL,
  param_emb_pool_type = NULL,
  param_aggregation = NULL,
  embeddings
)

Arguments

Returns

Returns an object of class EmbeddedText which stores the text embeddings produced by an objects of class TextEmbeddingModel.

Method save()

Saves a data set to disk.

Usage

EmbeddedText$save(dir_path, folder_name, create_dir = TRUE)

Arguments

Returns

Method does not return anything. It write the data set to disk.

Method is_configured()

Method for checking if the model was successfully configured. An object can only be used if this value is TRUE.

Usage

EmbeddedText$is_configured()

Returns

bool TRUE if the model is fully configured. FALSE if not.

Method load_from_disk()

loads an object of class EmbeddedText from disk and updates the object to the current version of the package.

Usage

EmbeddedText$load_from_disk(dir_path)

Arguments

Returns

Method does not return anything. It loads an object from disk.

Method get_model_info()

Method for retrieving information about the model that generated this embedding.

Usage

EmbeddedText$get_model_info()

Returns

list contains all saved information about the underlying text embedding model.

Method get_model_label()

Method for retrieving the label of the model that generated this embedding.

Usage

EmbeddedText$get_model_label()

Returns

string Label of the corresponding text embedding model

Method get_times()

Number of chunks/times of the text embeddings.

Usage

EmbeddedText$get_times()

Returns

Returns an int describing the number of chunks/times of the text embeddings.

Method get_features()

Number of actual features/dimensions of the text embeddings.In the case a feature extractor was used the number of features is smaller as the original number of features. To receive the original number of features (the number of features before applying a feature extractor) you can use the method get_original_features of this class.

Usage

EmbeddedText$get_features()

Returns

Returns an int describing the number of features/dimensions of the text embeddings.

Method get_original_features()

Number of original features/dimensions of the text embeddings.

Usage

EmbeddedText$get_original_features()

Returns

Returns an int describing the number of features/dimensions if no feature extractor) is used or before a feature extractor) is applied.

Method is_compressed()

Checks if the text embedding were reduced by a feature extractor.

Usage

EmbeddedText$is_compressed()

Returns

Returns TRUE if the number of dimensions was reduced by a feature extractor. If not return FALSE.

Method add_feature_extractor_info()

Method setting information on the feature extractor that was used to reduce the number of dimensions of the text embeddings. This information should only be used if a feature extractor was applied.

Usage

EmbeddedText$add_feature_extractor_info(
  model_name,
  model_label = NA,
  features = NA,
  method = NA,
  noise_factor = NA,
  optimizer = NA
)

Arguments

Returns

Method does nothing return. It sets information on a feature extractor.

Method get_feature_extractor_info()

Method for receiving information on the feature extractor that was used to reduce the number of dimensions of the text embeddings.

Usage

EmbeddedText$get_feature_extractor_info()

Returns

Returns a list with information on the feature extractor. If no feature extractor was used it returns NULL.

Method convert_to_LargeDataSetForTextEmbeddings()

Method for converting this object to an object of class LargeDataSetForTextEmbeddings.

Usage

EmbeddedText$convert_to_LargeDataSetForTextEmbeddings()

Returns

Returns an object of class LargeDataSetForTextEmbeddings which uses memory mapping allowing to work with large data sets.

Method n_rows()

Number of rows.

Usage

EmbeddedText$n_rows()

Returns

Returns the number of rows of the text embeddings which represent the number of cases.

Method get_all_fields()

Return all fields.

Usage

EmbeddedText$get_all_fields()

Returns

Method returns a list containing all public and private fields of the object.

Method clone()

The objects of this class are cloneable with this method.

Usage

EmbeddedText$clone(deep = FALSE)

Arguments

See Also

Other Data Management: DataManagerClassifier, LargeDataSetForText, LargeDataSetForTextEmbeddings

Abstract base class for large data sets

Description

This object contains public and private methods which may be useful for every large data sets. Objects of this class are not intended to be used directly. LargeDataSetForTextEmbeddings or LargeDataSetForText.

Value

Returns a new object of this class.

Methods

Public methods

• LargeDataSetBase$n_cols()

• LargeDataSetBase$n_rows()

• LargeDataSetBase$get_colnames()

• LargeDataSetBase$get_dataset()

• LargeDataSetBase$reduce_to_unique_ids()

• LargeDataSetBase$select()

• LargeDataSetBase$get_ids()

• LargeDataSetBase$save()

• LargeDataSetBase$load_from_disk()

• LargeDataSetBase$load()

• LargeDataSetBase$get_all_fields()

• LargeDataSetBase$clone()

Method n_cols()

Number of columns in the data set.

Usage

LargeDataSetBase$n_cols()

Returns

int describing the number of columns in the data set.

Method n_rows()

Number of rows in the data set.

Usage

LargeDataSetBase$n_rows()

Returns

int describing the number of rows in the data set.

Method get_colnames()

Get names of the columns in the data set.

Usage

LargeDataSetBase$get_colnames()

Returns

vector containing the names of the columns as strings.

Method get_dataset()

Get data set.

Usage

LargeDataSetBase$get_dataset()

Returns

Returns the data set of this object as an object of class datasets.arrow_dataset.Dataset.

Method reduce_to_unique_ids()

Reduces the data set to a data set containing only unique ids. In the case an id exists multiple times in the data set the first case remains in the data set. The other cases are dropped.

Attention Calling this method will change the data set in place.

Usage

LargeDataSetBase$reduce_to_unique_ids()

Returns

Method does not return anything. It changes the data set of this object in place.

Method select()

Returns a data set which contains only the cases belonging to the specific indices.

Usage

LargeDataSetBase$select(indicies)

Arguments

Returns

Returns a data set of class datasets.arrow_dataset.Dataset with the selected rows.

Method get_ids()

Get ids

Usage

LargeDataSetBase$get_ids()

Returns

Returns a vector containing the ids of every row as strings.

Method save()

Saves a data set to disk.

Usage

LargeDataSetBase$save(dir_path, folder_name, create_dir = TRUE)

Arguments

Returns

Method does not return anything. It write the data set to disk.

Method load_from_disk()

loads an object of class LargeDataSetBase from disk 'and updates the object to the current version of the package.

Usage

LargeDataSetBase$load_from_disk(dir_path)

Arguments

Returns

Method does not return anything. It loads an object from disk.

Method load()

Loads a data set from disk.

Usage

LargeDataSetBase$load(dir_path)

Arguments

Returns

Method does not return anything. It loads a data set from disk.

Method get_all_fields()

Return all fields.

Usage

LargeDataSetBase$get_all_fields()

Returns

Method returns a list containing all public and private fields of the object.

Method clone()

The objects of this class are cloneable with this method.

Usage

LargeDataSetBase$clone(deep = FALSE)

Arguments

Abstract class for large data sets containing raw texts

Description

This object stores raw texts. The data of this objects is not stored in memory directly. By using memory mapping these objects allow to work with data sets which do not fit into memory/RAM.

Value

Returns a new object of this class.

Super class

aifeducation::LargeDataSetBase -> LargeDataSetForText

Methods

Public methods

• LargeDataSetForText$new()

• LargeDataSetForText$add_from_files_txt()

• LargeDataSetForText$add_from_files_pdf()

• LargeDataSetForText$add_from_files_xlsx()

• LargeDataSetForText$add_from_data.frame()

• LargeDataSetForText$get_private()

• LargeDataSetForText$clone()

Method new()

Method for creation of LargeDataSetForText instance. It can be initialized with init_data parameter if passed (Uses add_from_data.frame() method if init_data is data.frame).

Usage

LargeDataSetForText$new(init_data = NULL)

Arguments

Returns

A new instance of this class initialized with init_data if passed.

Method add_from_files_txt()

Method for adding raw texts saved within .txt files to the data set. Please note the the directory should contain one folder for each .txt file. In order to create an informative data set every folder can contain the following additional files:

• bib_entry.txt: containing a text version of the bibliographic information of the raw text.

• license.txt: containing a statement about the license to use the raw text such as "CC BY".

• url_license.txt: containing the url/link to the license in the internet.

• text_license.txt: containing the license in raw text.

• url_source.txt: containing the url/link to the source in the internet.

  The id of every .txt file is the file name without file extension. Please be aware to provide unique file names. Id and raw texts are mandatory, bibliographic and license information are optional.

Usage

LargeDataSetForText$add_from_files_txt(
  dir_path,
  batch_size = 500,
  log_file = NULL,
  log_write_interval = 2,
  log_top_value = 0,
  log_top_total = 1,
  log_top_message = NA,
  trace = TRUE
)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method add_from_files_pdf()

Method for adding raw texts saved within .pdf files to the data set. Please note the the directory should contain one folder for each .pdf file. In order to create an informative data set every folder can contain the following additional files:

• bib_entry.txt: containing a text version of the bibliographic information of the raw text.

• license.txt: containing a statement about the license to use the raw text such as "CC BY".

• url_license.txt: containing the url/link to the license in the internet.

• text_license.txt: containing the license in raw text.

• url_source.txt: containing the url/link to the source in the internet.

  The id of every .pdf file is the file name without file extension. Please be aware to provide unique file names. Id and raw texts are mandatory, bibliographic and license information are optional.

Usage

LargeDataSetForText$add_from_files_pdf(
  dir_path,
  batch_size = 500,
  log_file = NULL,
  log_write_interval = 2,
  log_top_value = 0,
  log_top_total = 1,
  log_top_message = NA,
  trace = TRUE
)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method add_from_files_xlsx()

Method for adding raw texts saved within .xlsx files to the data set. The method assumes that the texts are saved in the rows and that the columns store the id and the raw texts in the columns. In addition, a column for the bibliography information and the license can be added. The column names for these rows must be specified with the following arguments. They must be the same for all .xlsx files in the chosen directory. Id and raw texts are mandatory, bibliographic, license, license's url, license's text, and source's url are optional. Additional columns are dropped.

Usage

LargeDataSetForText$add_from_files_xlsx(
  dir_path,
  trace = TRUE,
  id_column = "id",
  text_column = "text",
  bib_entry_column = "bib_entry",
  license_column = "license",
  url_license_column = "url_license",
  text_license_column = "text_license",
  url_source_column = "url_source",
  log_file = NULL,
  log_write_interval = 2,
  log_top_value = 0,
  log_top_total = 1,
  log_top_message = NA
)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method add_from_data.frame()

Method for adding raw texts from a data.frame

Usage

LargeDataSetForText$add_from_data.frame(data_frame)

Arguments

Returns

The method does not return anything. It adds new raw texts to the data set.

Method get_private()

Method for requesting all private fields and methods. Used for loading and updating an object.

Usage

LargeDataSetForText$get_private()

Returns

Returns a list with all private fields and methods.

Method clone()

The objects of this class are cloneable with this method.

Usage

LargeDataSetForText$clone(deep = FALSE)

Arguments

See Also

Other Data Management: DataManagerClassifier, EmbeddedText, LargeDataSetForTextEmbeddings

Abstract class for large data sets containing text embeddings

Description

This object stores text embeddings which are usually produced by an object of class TextEmbeddingModel. The data of this objects is not stored in memory directly. By using memory mapping these objects allow to work with data sets which do not fit into memory/RAM.

LargeDataSetForTextEmbeddings are used for storing and managing the text embeddings created with objects of class TextEmbeddingModel. Objects of class LargeDataSetForTextEmbeddings serve as input for objects of class TEClassifierRegular, TEClassifierProtoNet, and TEFeatureExtractor. The main aim of this class is to provide a structured link between embedding models and classifiers. Since objects of this class save information on the text embedding model that created the text embedding it ensures that only embedding generated with same embedding model are combined. Furthermore, the stored information allows objects to check if embeddings of the correct text embedding model are used for training and predicting.

Value

Returns a new object of this class.

Super class

aifeducation::LargeDataSetBase -> LargeDataSetForTextEmbeddings

Methods

Public methods

• LargeDataSetForTextEmbeddings$configure()

• LargeDataSetForTextEmbeddings$is_configured()

• LargeDataSetForTextEmbeddings$get_text_embedding_model_name()

• LargeDataSetForTextEmbeddings$get_model_info()

• LargeDataSetForTextEmbeddings$load_from_disk()

• LargeDataSetForTextEmbeddings$get_model_label()

• LargeDataSetForTextEmbeddings$add_feature_extractor_info()

• LargeDataSetForTextEmbeddings$get_feature_extractor_info()

• LargeDataSetForTextEmbeddings$is_compressed()

• LargeDataSetForTextEmbeddings$get_times()

• LargeDataSetForTextEmbeddings$get_features()

• LargeDataSetForTextEmbeddings$get_original_features()

• LargeDataSetForTextEmbeddings$add_embeddings_from_array()

• LargeDataSetForTextEmbeddings$add_embeddings_from_EmbeddedText()

• LargeDataSetForTextEmbeddings$add_embeddings_from_LargeDataSetForTextEmbeddings()

• LargeDataSetForTextEmbeddings$convert_to_EmbeddedText()

• LargeDataSetForTextEmbeddings$clone()

Method configure()

Creates a new object representing text embeddings.

Usage

LargeDataSetForTextEmbeddings$configure(
  model_name = NA,
  model_label = NA,
  model_date = NA,
  model_method = NA,
  model_version = NA,
  model_language = NA,
  param_seq_length = NA,
  param_chunks = NULL,
  param_features = NULL,
  param_overlap = NULL,
  param_emb_layer_min = NULL,
  param_emb_layer_max = NULL,
  param_emb_pool_type = NULL,
  param_aggregation = NULL
)

Arguments

Returns

The method returns a new object of this class.

Method is_configured()

Method for checking if the model was successfully configured. An object can only be used if this value is TRUE.

Usage

LargeDataSetForTextEmbeddings$is_configured()

Returns

bool TRUE if the model is fully configured. FALSE if not.

Method get_text_embedding_model_name()

Method for requesting the name (unique id) of the underlying text embedding model.

Usage

LargeDataSetForTextEmbeddings$get_text_embedding_model_name()

Returns

Returns a string describing name of the text embedding model.

Method get_model_info()

Method for retrieving information about the model that generated this embedding.

Usage

LargeDataSetForTextEmbeddings$get_model_info()

Returns

list containing all saved information about the underlying text embedding model.

Method load_from_disk()

loads an object of class LargeDataSetForTextEmbeddings from disk and updates the object to the current version of the package.

Usage

LargeDataSetForTextEmbeddings$load_from_disk(dir_path)

Arguments

Returns

Method does not return anything. It loads an object from disk.

Method get_model_label()

Method for retrieving the label of the model that generated this embedding.

Usage

LargeDataSetForTextEmbeddings$get_model_label()

Returns

string Label of the corresponding text embedding model

Method add_feature_extractor_info()

Method setting information on the TEFeatureExtractor that was used to reduce the number of dimensions of the text embeddings. This information should only be used if a TEFeatureExtractor was applied.

Usage

LargeDataSetForTextEmbeddings$add_feature_extractor_info(
  model_name,
  model_label = NA,
  features = NA,
  method = NA,
  noise_factor = NA,
  optimizer = NA
)

Arguments

Returns

Method does nothing return. It sets information on a TEFeatureExtractor.

Method get_feature_extractor_info()

Method for receiving information on the TEFeatureExtractor that was used to reduce the number of dimensions of the text embeddings.

Usage

LargeDataSetForTextEmbeddings$get_feature_extractor_info()

Returns

Returns a list with information on the TEFeatureExtractor. If no TEFeatureExtractor was used it returns NULL.

Method is_compressed()

Checks if the text embedding were reduced by a TEFeatureExtractor.

Usage

LargeDataSetForTextEmbeddings$is_compressed()

Returns

Returns TRUE if the number of dimensions was reduced by a TEFeatureExtractor. If not return FALSE.

Method get_times()

Number of chunks/times of the text embeddings.

Usage

LargeDataSetForTextEmbeddings$get_times()

Returns

Returns an int describing the number of chunks/times of the text embeddings.

Method get_features()

Number of actual features/dimensions of the text embeddings.In the case a TEFeatureExtractor was used the number of features is smaller as the original number of features. To receive the original number of features (the number of features before applying a TEFeatureExtractor) you can use the method get_original_features of this class.

Usage

LargeDataSetForTextEmbeddings$get_features()

Returns

Returns an int describing the number of features/dimensions of the text embeddings.

Method get_original_features()

Number of original features/dimensions of the text embeddings.

Usage

LargeDataSetForTextEmbeddings$get_original_features()

Returns

Returns an int describing the number of features/dimensions if no TEFeatureExtractor) is used or before a TEFeatureExtractor) is applied.

Method add_embeddings_from_array()

Method for adding new data to the data set from an array. Please note that the method does not check if cases already exist in the data set. To reduce the data set to unique cases call the method reduce_to_unique_ids.

Usage

LargeDataSetForTextEmbeddings$add_embeddings_from_array(embedding_array)

Arguments

Returns

The method does not return anything. It adds new data to the data set.

Method add_embeddings_from_EmbeddedText()

Method for adding new data to the data set from an EmbeddedText. Please note that the method does not check if cases already exist in the data set. To reduce the data set to unique cases call the method reduce_to_unique_ids.

Usage

LargeDataSetForTextEmbeddings$add_embeddings_from_EmbeddedText(EmbeddedText)

Arguments

Returns

The method does not return anything. It adds new data to the data set.

Method add_embeddings_from_LargeDataSetForTextEmbeddings()

Method for adding new data to the data set from an LargeDataSetForTextEmbeddings. Please note that the method does not check if cases already exist in the data set. To reduce the data set to unique cases call the method reduce_to_unique_ids.

Usage

LargeDataSetForTextEmbeddings$add_embeddings_from_LargeDataSetForTextEmbeddings(
  dataset
)

Arguments

Returns

The method does not return anything. It adds new data to the data set.

Method convert_to_EmbeddedText()

Method for converting this object to an object of class EmbeddedText.

Attention This object uses memory mapping to allow the usage of data sets that do not fit into memory. By calling this method the data set will be loaded and stored into memory/RAM. This may lead to an out-of-memory error.

Usage

LargeDataSetForTextEmbeddings$convert_to_EmbeddedText()

Returns

LargeDataSetForTextEmbeddings an object of class EmbeddedText which is stored in the memory/RAM.

Method clone()

The objects of this class are cloneable with this method.

Usage

LargeDataSetForTextEmbeddings$clone(deep = FALSE)

Arguments

See Also

Other Data Management: DataManagerClassifier, EmbeddedText, LargeDataSetForText

Server function for: graphical user interface for showing the license.

Description

Functions generates the functionality of a page on the server.

Usage

License_Server(id)

Arguments

Value

This function does nothing return. It is used to create the functionality of a page for a shiny app.

Server function for: graphical user interface for displaying the reliability of classifiers.

Description

Functions generates the functionality of a page on the server.

Usage

Reliability_Server(id, model)

Arguments

Value

This function does nothing return. It is used to create the functionality of a page for a shiny app.

See Also

Other studio_gui_page_classifier_reliability: Reliability_UI()

Graphical user interface for displaying the reliability of classifiers.

Description

Functions generates the tab within a page for displaying infomration on the reliability of classifiers.

Usage

Reliability_UI(id)

Arguments

Value

This function does nothing return. It is used to build a page for a shiny app.

See Also

Other studio_gui_page_classifier_reliability: Reliability_Server()

Text embedding classifier with a ProtoNet

Description

Abstract class for neural nets with 'keras'/'tensorflow' and 'pytorch'.

This object represents in implementation of a prototypical network for few-shot learning as described by Snell, Swersky, and Zemel (2017). The network uses a multi way contrastive loss described by Zhang et al. (2019). The network learns to scale the metric as described by Oreshkin, Rodriguez, and Lacoste (2018)

Value

Objects of this class are used for assigning texts to classes/categories. For the creation and training of a classifier an object of class EmbeddedText or LargeDataSetForTextEmbeddings and a factor are necessary. The object of class EmbeddedText or LargeDataSetForTextEmbeddings contains the numerical text representations (text embeddings) of the raw texts generated by an object of class TextEmbeddingModel. The factor contains the classes/categories for every text. Missing values (unlabeled cases) are supported. For predictions an object of class EmbeddedText or LargeDataSetForTextEmbeddings has to be used which was created with the same TextEmbeddingModel as for training.

Super classes

aifeducation::AIFEBaseModel -> aifeducation::TEClassifierRegular -> TEClassifierProtoNet

Methods

Public methods

• TEClassifierProtoNet$configure()

• TEClassifierProtoNet$train()

• TEClassifierProtoNet$embed()

• TEClassifierProtoNet$plot_embeddings()

• TEClassifierProtoNet$clone()

Method configure()

Creating a new instance of this class.

Usage

TEClassifierProtoNet$configure(
  ml_framework = "pytorch",
  name = NULL,
  label = NULL,
  text_embeddings = NULL,
  feature_extractor = NULL,
  target_levels = NULL,
  dense_size = 4,
  dense_layers = 0,
  rec_size = 4,
  rec_layers = 2,
  rec_type = "gru",
  rec_bidirectional = FALSE,
  embedding_dim = 2,
  self_attention_heads = 0,
  intermediate_size = NULL,
  attention_type = "fourier",
  add_pos_embedding = TRUE,
  rec_dropout = 0.1,
  repeat_encoder = 1,
  dense_dropout = 0.4,
  recurrent_dropout = 0.4,
  encoder_dropout = 0.1,
  optimizer = "adam"
)

Arguments

Returns

Returns an object of class TEClassifierProtoNet which is ready for training.

Method train()

Method for training a neural net.

Training includes a routine for early stopping. In the case that loss<0.0001 and Accuracy=1.00 and Average Iota=1.00 training stops. The history uses the values of the last trained epoch for the remaining epochs.

After training the model with the best values for Average Iota, Accuracy, and Loss on the validation data set is used as the final model.

Usage

TEClassifierProtoNet$train(
  data_embeddings,
  data_targets,
  data_folds = 5,
  data_val_size = 0.25,
  use_sc = TRUE,
  sc_method = "dbsmote",
  sc_min_k = 1,
  sc_max_k = 10,
  use_pl = TRUE,
  pl_max_steps = 3,
  pl_max = 1,
  pl_anchor = 1,
  pl_min = 0,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  epochs = 40,
  batch_size = 35,
  Ns = 5,
  Nq = 3,
  loss_alpha = 0.5,
  loss_margin = 0.5,
  sampling_separate = FALSE,
  sampling_shuffle = TRUE,
  dir_checkpoint,
  trace = TRUE,
  ml_trace = 1,
  log_dir = NULL,
  log_write_interval = 10,
  n_cores = auto_n_cores()
)

Arguments

Details

• sc_max_k: All values from sc_min_k up to sc_max_k are successively used. If the number of sc_max_k is too high, the value is reduced to a number that allows the calculating of synthetic units.

• ⁠pl_anchor:⁠ With the help of this value, the new cases are sorted. For this aim, the distance from the anchor is calculated and all cases are arranged into an ascending order.

Returns

Function does not return a value. It changes the object into a trained classifier.

Method embed()

Method for embedding documents. Please do not confuse this type of embeddings with the embeddings of texts created by an object of class TextEmbeddingModel. These embeddings embed documents according to their similarity to specific classes.

Usage

TEClassifierProtoNet$embed(embeddings_q = NULL, batch_size = 32)

Arguments

Returns

Returns a list containing the following elements

• embeddings_q: embeddings for the cases (query sample).

• embeddings_prototypes: embeddings of the prototypes which were learned during training. They represents the center for the different classes.

Method plot_embeddings()

Method for creating a plot to visualize embeddings and their corresponding centers (prototypes).

Usage

TEClassifierProtoNet$plot_embeddings(
  embeddings_q,
  classes_q = NULL,
  batch_size = 12,
  alpha = 0.5,
  size_points = 3,
  size_points_prototypes = 8,
  inc_unlabeled = TRUE
)

Arguments

Returns

Returns a plot of class ggplotvisualizing embeddings.

Method clone()

The objects of this class are cloneable with this method.

Usage

TEClassifierProtoNet$clone(deep = FALSE)

Arguments

References

Oreshkin, B. N., Rodriguez, P. & Lacoste, A. (2018). TADAM: Task dependent adaptive metric for improved few-shot learning. https://doi.org/10.48550/arXiv.1805.10123

Snell, J., Swersky, K. & Zemel, R. S. (2017). Prototypical Networks for Few-shot Learning. https://doi.org/10.48550/arXiv.1703.05175

Zhang, X., Nie, J., Zong, L., Yu, H. & Liang, W. (2019). One Shot Learning with Margin. In Q. Yang, Z.-H. Zhou, Z. Gong, M.-L. Zhang & S.-J. Huang (Eds.), Lecture Notes in Computer Science. Advances in Knowledge Discovery and Data Mining (Vol. 11440, pp. 305–317). Springer International Publishing. https://doi.org/10.1007/978-3-030-16145-3_24

See Also

Other Classification: TEClassifierRegular

Text embedding classifier with a neural net

Description

Abstract class for neural nets with 'keras'/'tensorflow' and ' pytorch'.

Value

Objects of this class are used for assigning texts to classes/categories. For the creation and training of a classifier an object of class EmbeddedText or LargeDataSetForTextEmbeddings on the one hand and a factor on the other hand are necessary.

The object of class EmbeddedText or LargeDataSetForTextEmbeddings contains the numerical text representations (text embeddings) of the raw texts generated by an object of class TextEmbeddingModel. For supporting large data sets it is recommended to use LargeDataSetForTextEmbeddings instead of EmbeddedText.

The factor contains the classes/categories for every text. Missing values (unlabeled cases) are supported and can be used for pseudo labeling.

For predictions an object of class EmbeddedText or LargeDataSetForTextEmbeddings has to be used which was created with the same TextEmbeddingModel as for training.

Super class

aifeducation::AIFEBaseModel -> TEClassifierRegular

Public fields

Methods

Public methods

• TEClassifierRegular$configure()

• TEClassifierRegular$train()

• TEClassifierRegular$predict()

• TEClassifierRegular$check_embedding_model()

• TEClassifierRegular$check_feature_extractor_object_type()

• TEClassifierRegular$requires_compression()

• TEClassifierRegular$save()

• TEClassifierRegular$load_from_disk()

• TEClassifierRegular$clone()

Method configure()

Creating a new instance of this class.

Usage

TEClassifierRegular$configure(
  ml_framework = "pytorch",
  name = NULL,
  label = NULL,
  text_embeddings = NULL,
  feature_extractor = NULL,
  target_levels = NULL,
  dense_size = 4,
  dense_layers = 0,
  rec_size = 4,
  rec_layers = 2,
  rec_type = "gru",
  rec_bidirectional = FALSE,
  self_attention_heads = 0,
  intermediate_size = NULL,
  attention_type = "fourier",
  add_pos_embedding = TRUE,
  rec_dropout = 0.1,
  repeat_encoder = 1,
  dense_dropout = 0.4,
  recurrent_dropout = 0.4,
  encoder_dropout = 0.1,
  optimizer = "adam"
)

Arguments

Returns

Returns an object of class TEClassifierRegular which is ready for training.

Method train()

Method for training a neural net.

Training includes a routine for early stopping. In the case that loss<0.0001 and Accuracy=1.00 and Average Iota=1.00 training stops. The history uses the values of the last trained epoch for the remaining epochs.

After training the model with the best values for Average Iota, Accuracy, and Loss on the validation data set is used as the final model.

Usage

TEClassifierRegular$train(
  data_embeddings,
  data_targets,
  data_folds = 5,
  data_val_size = 0.25,
  balance_class_weights = TRUE,
  balance_sequence_length = TRUE,
  use_sc = TRUE,
  sc_method = "dbsmote",
  sc_min_k = 1,
  sc_max_k = 10,
  use_pl = TRUE,
  pl_max_steps = 3,
  pl_max = 1,
  pl_anchor = 1,
  pl_min = 0,
  sustain_track = TRUE,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 15,
  epochs = 40,
  batch_size = 32,
  dir_checkpoint,
  trace = TRUE,
  ml_trace = 1,
  log_dir = NULL,
  log_write_interval = 10,
  n_cores = auto_n_cores()
)

Arguments