`i!bddlmZddlmZmZGddeZGddeZddgZdS) )PretrainedConfig)CONFIG_MAPPING AutoConfigcNeZdZdZdZddddddZ dfd ZxZS)VoxtralEncoderConfiga` This is the configuration class to store the configuration of a [`VoxtralEncoder`]. It is used to instantiate a Voxtral audio encoder according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the audio encoder of the Voxtral architecture. e.g. [mistralai/Voxtral-Mini-3B-2507](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507) Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 51866): Vocabulary size of the model. hidden_size (`int`, *optional*, defaults to 1280): Dimensionality of the hidden representations. intermediate_size (`int`, *optional*, defaults to 5120): Dimension of the MLP representations. num_hidden_layers (`int`, *optional*, defaults to 32): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 20): Number of attention heads for each attention layer in the Transformer encoder. scale_embedding (`bool`, *optional*, defaults to `False`): Scale embeddings by dividing by sqrt(hidden_size) if True. activation_function (`str`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, "gelu", num_mel_bins (`int`, *optional*, defaults to 128): Number of mel features used per input features. Should correspond to the value used in the `VoxtralProcessor` class. max_source_positions (`int`, *optional*, defaults to 1500): The maximum sequence length of log-mel filter-bank features that this model might ever be used with. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. ```python >>> from transformers import VoxtralEncoderConfig, VoxtralEncoder >>> # Initializing a VoxtralEncoderConfig >>> configuration = VoxtralEncoderConfig() >>> # Initializing a VoxtralEncoder (with random weights) >>> model = VoxtralEncoder(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```voxtral_encoder hidden_sizenum_hidden_layersnum_attention_headsintermediate_size layerdrop)d_modelencoder_layersencoder_attention_headsencoder_ffn_dimencoder_layerdrop Fgelu{Gz?c  tjdi| ||_||_||_||_||_||_||_||_ | |_ | |_ d|_ d|_ d|_| |_dS)Nr)super__init__ vocab_sizer r r r scale_embeddingactivation_function num_mel_binsmax_source_positionsinitializer_rangedropoutractivation_dropoutattention_dropout)selfr"r r r r r#r$r%r&r'r*kwargs __class__s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/voxtral/configuration_voxtral.pyr!zVoxtralEncoderConfig.__init__Ps ""6"""$&!2!2#6 .#6 ($8!!2  "%!2) rrrrrFrrrrr)__name__ __module__ __qualname____doc__ model_type attribute_mapr! __classcell__r-s@r.rrs/ / b#J!-#8.( M"!$3$3$3$3$3$3$3$3$3$3r/rc TeZdZdZdZeedZddddddd d d d d Z dfd ZxZ S) VoxtralConfiga3 This is the configuration class to store the configuration of a [`VoxtralForConditionalGeneration`]. It is used to instantiate an Voxtral model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the Voxtral-Mini-3B. e.g. [mistralai/Voxtral-Mini-3B-2507](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507) Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: audio_config (`Union[AutoConfig, dict]`, *optional*): The config object or dictionary of the audio encoder. text_config (`Union[AutoConfig, dict]`, *optional*): The config object or dictionary of the text model. audio_token_id (`int`, *optional*): The image token index to encode the image prompt. projector_hidden_act (`str`, *optional*, defaults to `"gelu"`): The activation function (function or string) in the multi-modal projector. ```python >>> from transformers import VoxtralForConditionalGeneration, VoxtralConfig >>> # Initializing a Voxtral configuration >>> configuration = VoxtralConfig(audio_token_id=24, projector_hidden_act="gelu") >>> # Initializing a 3B model with random weights >>> model = VoxtralForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```voxtral) text_config audio_configii i gh㈵>TgחAr) r"r r r num_key_value_headsmax_position_embeddings rms_norm_eps use_cache rope_thetahead_dimNrc .t|tr2|dd|d<t|ddi|}n|td}||_t|tr;|dd|d<t|ddii|j|}n|tddi|j}||_|j|_|j|_||_ ||_ tj di|dS)Nr4r llamar) isinstancedictgetrr<_default_text_config_kwargsr;r"r audio_token_idprojector_hidden_actr r!)r+r<r;rKrLr,r-s r.r!zVoxtralConfig.__init__sC lD ) ) ?)5)9)9,HY)Z)ZL &),|*DEUU UULL  !)*;<>>L( k4 ( ( V(3 g(N(NK %(\)BCET5EEKK (1UUD4TUUK&%0&2,$8!""6"""""r/)NNNr) r0r1r2r3r4r sub_configsrJr!r6r7s@r.r9r9ws  BJ",jIIK! #)! # ## ##########r/r9N)configuration_utilsrautorrrr9__all__rr/r.rQs 433333--------`3`3`3`3`3+`3`3`3FO#O#O#O#O#$O#O#O#d "? 3r/