src.models.auto_clm

auto_clm.py

Default Causal Language Model (CLM) & Tokenizer Specification and Initialization. Downloads Model Configuration (if necessary) from the Hugging Face transformers Hub, instantiates pretrained Tokenizer, and initializes model using the necessary AutoModel class.

Functions

get_auto_clm_tokenizer

Download/Load AutoConfig and Instantiate Corresponding Model and Tokenizer.