`i#BddlmZddlmZddlmZddlmZmZej e Z GddeZ Gdd eZ Gd d eZgd Zd S))PretrainedConfig)!MODEL_FOR_CAUSAL_LM_MAPPING_NAMES)logging)CONFIG_MAPPING AutoConfigcBeZdZdZdZdZ dfd ZxZS)InstructBlipVideoVisionConfiga This is the configuration class to store the configuration of a [`InstructBlipVideoVisionModel`]. It is used to instantiate a InstructBlipVideo vision encoder according to the specified arguments, defining the model architecture. Instantiating a configuration defaults will yield a similar configuration to that of the InstructBlipVideo [Salesforce/instruct-blip-flan-t5](https://huggingface.co/Salesforce/instruct-blip-flan-t5) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: hidden_size (`int`, *optional*, defaults to 1408): Dimensionality of the encoder layers and the pooler layer. intermediate_size (`int`, *optional*, defaults to 6144): Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. num_hidden_layers (`int`, *optional*, defaults to 39): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer encoder. image_size (`int`, *optional*, defaults to 224): The size (resolution) of each image. patch_size (`int`, *optional*, defaults to 14): The size (resolution) of each patch. hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"selu"` and `"gelu_new"` `"gelu"` are supported. to 1e-5): The epsilon used by the layer normalization layers. layer_norm_eps (`float`, *optional*, defaults to 1e-06): The epsilon used by the layer normalization layers. attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. initializer_range (`float`, *optional*, defaults to 1e-10): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. qkv_bias (`bool`, *optional*, defaults to `True`): Whether to add a bias to the queries and values in the self-attention layers. Example: ```python >>> from transformers import InstructBlipVideoVisionConfig, InstructBlipVideoVisionModel >>> # Initializing a InstructBlipVideoVisionConfig with Salesforce/instruct-blip-flan-t5 style configuration >>> configuration = InstructBlipVideoVisionConfig() >>> # Initializing a InstructBlipVideoVisionModel (with random weights) from the Salesforce/instruct-blip-flan-t5 style configuration >>> model = InstructBlipVideoVisionModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```instructblipvideo_vision_model vision_config'geluư>绽|=Tc  tjdi| ||_||_||_||_||_||_| |_| |_ ||_ ||_ | |_ dS)N) super__init__ hidden_sizeintermediate_sizenum_hidden_layersnum_attention_heads patch_size image_sizeinitializer_rangeattention_dropoutlayer_norm_eps hidden_actqkv_bias)selfrrrrr rr$r#r"r!r%kwargs __class__s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/instructblipvideo/configuration_instructblipvideo.pyrz&InstructBlipVideoVisionConfig.__init__Vsz ""6"""&!2!2#6 $$!2!2,$  ) r rrrrrrrrrT__name__ __module__ __qualname____doc__ model_typebase_config_keyr __classcell__r(s@r)r r sw0 0 d2J%O!!!!!!!!!!r*r cJeZdZdZdZdZ dfd ZxZS)InstructBlipVideoQFormerConfiga This is the configuration class to store the configuration of a [`InstructBlipVideoQFormerModel`]. It is used to instantiate a InstructBlipVideo Querying Transformer (Q-Former) model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the InstructBlipVideo [Salesforce/instruct-blip-flan-t5](https://huggingface.co/Salesforce/instruct-blip-flan-t5) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Note that [`InstructBlipVideoQFormerModel`] is very similar to [`BertLMHeadModel`] with interleaved cross-attention. Args: vocab_size (`int`, *optional*, defaults to 30522): Vocabulary size of the Q-Former model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling the model. hidden_size (`int`, *optional*, defaults to 768): Dimensionality of the encoder layers and the pooler layer. num_hidden_layers (`int`, *optional*, defaults to 12): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 12): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 3072): Dimensionality of the "intermediate" (often named feed-forward) layer in the Transformer encoder. hidden_act (`str` or `Callable`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"silu"` and `"gelu_new"` are supported. hidden_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities. max_position_embeddings (`int`, *optional*, defaults to 512): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. layer_norm_eps (`float`, *optional*, defaults to 1e-12): The epsilon used by the layer normalization layers. pad_token_id (`int`, *optional*, defaults to 0): Token id used for padding sequences. position_embedding_type (`str`, *optional*, defaults to `"absolute"`): Type of position embedding. Choose one of `"absolute"`, `"relative_key"`, `"relative_key_query"`. For positional embeddings use `"absolute"`. For more information on `"relative_key"`, please refer to [Self-Attention with Relative Position Representations (Shaw et al.)](https://huggingface.co/papers/1803.02155). For more information on `"relative_key_query"`, please refer to *Method 4* in [Improve Transformer Models with Better Relative Position Embeddings (Huang et al.)](https://huggingface.co/papers/2009.13658). cross_attention_frequency (`int`, *optional*, defaults to 2): The frequency of adding cross-attention to the Transformer layers. encoder_hidden_size (`int`, *optional*, defaults to 1408): The hidden size of the hidden states for cross-attention. Examples: ```python >>> from transformers import InstructBlipVideoQFormerConfig, InstructBlipVideoQFormerModel >>> # Initializing a InstructBlipVideo Salesforce/instruct-blip-flan-t5 style configuration >>> configuration = InstructBlipVideoQFormerConfig() >>> # Initializing a model (with random weights) from the Salesforce/instruct-blip-flan-t5 style configuration >>> model = InstructBlipVideoQFormerModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```instructblipvideo_qformerqformer_config:w r皙?{Gz?-q=absoluterr c tjdd| i|||_||_||_||_||_||_||_||_ | |_ | |_ | |_ | |_ ||_||_dS)N pad_token_idr)rr vocab_sizerrrr$rhidden_dropout_probattention_probs_dropout_probmax_position_embeddingsr!r#position_embedding_typecross_attention_frequencyencoder_hidden_size)r&rDrrrrr$rErFrGr!r#rCrHrIrJr'r(s r)rz'InstructBlipVideoQFormerConfig.__init__s& ==l=f===$&!2#6 $!2#6 ,H)'>$!2,'>$)B   r*)r8r9r:r:r;rr<r<r=r>r?r@rArr r+r3s@r)r5r5ts= = ~-J&O%( # *"# !"7"7"7"7"7"7"7"7"7"7r*r5cjeZdZdZdZddiZeeedZ d fd Z e d ed ed e fd Z xZS)InstructBlipVideoConfiga [`InstructBlipVideoConfig`] is the configuration class to store the configuration of a [`InstructBlipVideoForConditionalGeneration`]. It is used to instantiate a Instructblipvideo model according to the specified arguments, defining the vision model, Q-Former model and language model configs. Instantiating a configuration with the defaults will yield a similar configuration to that of the Instructblipvideo [Salesforce/instruct-blip-flan-t5](https://huggingface.co/Salesforce/instruct-blip-flan-t5) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vision_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`InstructBlipVideoVisionConfig`]. qformer_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`InstructBlipVideoQFormerConfig`]. text_config (`dict`, *optional*): Dictionary of configuration options used to initialize any [`PretrainedConfig`]. num_query_tokens (`int`, *optional*, defaults to 32): The number of query tokens passed through the Transformer. video_token_index (`int`, *optional*): Token index of special video token. kwargs (*optional*): Dictionary of keyword arguments. Example: ```python >>> from transformers import ( ... InstructBlipVideoVisionConfig, ... InstructBlipVideoQFormerConfig, ... OPTConfig, ... InstructBlipVideoConfig, ... InstructBlipVideoForConditionalGeneration, ... ) >>> # Initializing a InstructBlipVideoConfig with Salesforce/instruct-blip-flan-t5 style configuration >>> configuration = InstructBlipVideoConfig() >>> # Initializing a InstructBlipVideoForConditionalGeneration (with random weights) from the Salesforce/instruct-blip-flan-t5 style configuration >>> model = InstructBlipVideoForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config >>> # We can also initialize a InstructBlipVideoConfig from a InstructBlipVideoVisionConfig, InstructBlipVideoQFormerConfig and any PretrainedConfig >>> # Initializing Instructblipvideo vision, Instructblipvideo Q-Former and language model configurations >>> vision_config = InstructBlipVideoVisionConfig() >>> qformer_config = InstructBlipVideoQFormerConfig() >>> text_config = OPTConfig() >>> config = InstructBlipVideoConfig.from_text_vision_configs(vision_config, qformer_config, text_config) ```instructblipvideovideo_token_idvideo_token_index) text_configr7r N c tjdi||i}td|i}td|i}tdt di||_t di||_|dd}t|di||_ ||_ ||_ |jj |j_|j jt v|_d|_d|_dS) NzZvision_config is None. initializing the InstructBlipVideoVisionConfig with default values.z\qformer_config is None. Initializing the InstructBlipVideoQFormerConfig with default values.zTtext_config is None. Initializing the text config with default values (`OPTConfig`).r0optg?r>r)rrloggerinfor r r5r7getrrPnum_query_tokensrOrrJr0ruse_decoder_only_language_modelinitializer_factorr!) r&r r7rPrWrOr'text_model_typer(s r)rz InstructBlipVideoConfig.__init__s! ""6"""  M KKt u u u  !N KKv w w w  K KKn o o o:KK]KK<NN~NN%//,>>)/:II[II 0!2262D2P//3/?/JNo/o,"%!%r*r r7rPc |d|||d|S)a  Instantiate a [`InstructBlipVideoConfig`] (or a derived class) from a InstructBlipVideo vision model, Q-Former and language model configurations. Returns: [`InstructBlipVideoConfig`]: An instance of a configuration object )r r7rPr)to_dict)clsr r7rPr's r) from_vision_qformer_text_configsz8InstructBlipVideoConfig.from_vision_qformer_text_configsAsY s '//11)1133#++--      r*)NNNrQN)r,r-r.r/r0 attribute_maprr5r sub_configsr classmethodrr^r2r3s@r)rLrLs5 5 n%J-M"86K !&!&!&!&!&!&F 4 7 &    [     r*rL)rLr5r N)configuration_utilsrmodels.auto.modeling_autorutilsrautorr get_loggerr,rTr r5rL__all__rr*r)rhs.433333JJJJJJ--------  H % %Q!Q!Q!Q!Q!$4Q!Q!Q!he7e7e7e7e7%5e7e7e7Pz z z z z .z z z z i h hr*