src.core.trainer.LMDataCollator

class LMDataCollator(tokenizer: transformers.tokenization_utils_base.PreTrainedTokenizerBase)[source]

Bases: object

Methods

Attributes

tokenizer

__call__(examples: List[BatchEncoding])[source]

Call self as a function.