`iǼdZddlmZmZmZddlZddlmZddlmZm Z m Z ddl m Z ddl mZmZdd lmZdd lmZdd lmZdd lmZdd lmZmZmZmZddlmZmZddl m!Z!ddl"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)ddl*m+Z+e&r ddl,m-Z-ddl.m/Z/e'j0e1Z2Gddej3Z4 d4dej5dej6dej6dej6deej6de7de7fd Z8Gd!d"ej5Z9Gd#d$eZ:e$Gd%d&eZ;Gd'd(e;Ze$d-.Gd/d0e;Z?e$Gd1d2e;Z@gd3ZAdS)5zPyTorch OPT model.)CallableOptionalUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)Cache DynamicCache)GenerationMixin)AttentionMaskConverter)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPast)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging)deprecate_kwarg) OPTConfig) BlockMask)make_flex_block_causal_maskcjeZdZdZdedeffd Z d dejded eejffd Z xZ S) OPTLearnedPositionalEmbeddingzN This module learns positional embeddings up to a fixed maximum size. num_embeddings embedding_dimcjd|_t||jz|dSN)offsetsuper__init__)selfr%r& __class__s x/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/opt/modeling_opt.pyr,z&OPTLearnedPositionalEmbedding.__init__8s3  $+5}EEEEErNattention_maskpast_key_values_length position_idsc|>tj|d}||zdz }|dd|df}t||jzS)z3`input_ids_shape` is expected to be [bsz x seqlen].Nrdim)torchcumsumlongr+forwardr*)r-r1r2r3r.s r/r:z%OPTLearnedPositionalEmbedding.forward>sq   <A>>>L(>9A=CCEEL'+A+B+B(BCLww|dk9:::r0)rN) __name__ __module__ __qualname____doc__intr,r7 LongTensorrr: __classcell__r.s@r/r$r$3sFsF3FFFFFF'(37 ;;(;!$;u/0 ;;;;;;;;;;r0r$modulequerykeyvaluer1scalingdropoutc tj||dd|z}|||z}tj|dtj|j}tj |||j }tj||} | dd } | |fS)N)r6dtypeptrainingrr)) r7matmul transposer functionalsoftmaxfloat32torMrIrP contiguous) rDrErFrGr1rHrIkwargs attn_weights attn_outputs r/eager_attention_forwardr[Ps<s}}R'<'<==GL!#n4 =((2U](SSVVW\WbccL=((6?([[L,|U33K''1--88::K  $$r0c*eZdZdZ ddedeeffd Zeddd  dd e j dee d ee j d ee j de dee j de e j ee j ee ffdZxZS) OPTAttentionz=Multi-headed attention from 'Attention Is All You Need' paperNconfig layer_idxc 4t||_|j|_|j|_|j|_|j |_ ||_ |(t d|j jd|j|jz|_d|_|j|jz|jkr t#d|jd|jd|jdz|_t'j|j|j|j |_t'j|j|j|j |_t'j|j|j|j |_t'j|j|j|j |_dS) NzInstantiating z without passing a `layer_idx` is not recommended and will lead to errors during the forward call if caching is used. Please make sure to provide a `layer_idx` when creating this class.Tz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩bias)r+r,r^ hidden_size embed_dimnum_attention_heads num_headsattention_dropoutrI enable_biasr_logger warning_oncer.r;head_dim is_causal ValueErrorrHrLineark_projv_projq_projout_proj)r-r^r_rXr.s r/r,zOPTAttention.__init__js  +3/ !-"     ,!8,,,    $.8  MDN *t~ = =8dn88%)^888 }d* iTEUVVV iTEUVVV iTEUVVV  $.$.tGWXXX r0past_key_valuepast_key_values4.58new_nameversionF hidden_statesr1layer_head_maskoutput_attentionscache_positionreturnc v|\}} } |||jz} | |d|j|jdd} ||} ||} | |d|j|jdd} | |d|j|jdd} |"| | | |j d|i\} } t}|j j dkrt|j j }||| | | |f|jsdn|jdd |\}}||| d}||}|sd}||fS) z#Input shape: Batch x Time x ChannelrKrr)Nr|eagerrC?)rIrH)sizerqrHviewrfrkrRrorpupdater_r[r^_attn_implementationrrPrIreshaperWrr)r-ryrtr1rzr{r|rXbsztgt_len_ query_states key_states value_statesattention_interfacerZrYs r/r:zOPTAttention.forwards(,,..Wa{{=11DL@ #((b$.$-PPZZ[\^_`` [[// {{=11 __S"dndmLLVVWXZ[\\ #((b$.$-PPZZ[\^_``  &'6'='=L$.;K^:\(( $J )@ ; +w 6 6"9$+:Z"[ $7$7      % $}>CC$, % %  % % ! \"))#w;;FFHH mmK00   LL((r0N)NNNFN)r;r<r=r>r rr?r,rr7Tensorr booltupler:rArBs@r/r]r]gs0GG $(!Y!Y!YC=!Y!Y!Y!Y!Y!YF_%0A6RRR,01526"'156)6)|6)"%6)!. 6) "%,/ 6)  6)!.6) u|Xel3Xe_D E6)6)6)SR6)6)6)6)6)r0r]czeZdZddedeeffd Zeddd dd ej d eej d eej dee d ee dee deej deej de edeejeeejejfffdZxZS)OPTDecoderLayerNr^r_c*t|j|_t |||_|j|_|j|_t|j |_ tj |j|j |_tj|j|j|j|_tj|j|j|j|_tj |j|j |_dS)N)r^r_elementwise_affinera)r+r,rcrdr] self_attndo_layer_norm_beforerIr activation_function activation_fnr LayerNormlayer_norm_elementwise_affineself_attn_layer_normrnffn_dimrhfc1fc2final_layer_norm)r-r^r_r.s r/r,zOPTDecoderLayer.__init__s +%VyIII$*$?!~ #F$>?$&L Nv/S% % % !9T^V^&BTUUU9V^T^&BTUUU " T^PVPt u u ur0rsrtrurvFryr1rzr{ use_cacher3r|rXr}c |} |jr||}|jd|||||||d| \}} tj||j|j}| |z}|js||}|j} |d| d}|} |jr| |}| |}| |}| |}tj||j|j}| |z| }|js| |}|f} |r| | fz } | S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`, *optional*): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_values (`Cache`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence.. )ryrtr3r1rzr{r|rNrK)rrrrrSrIrPshaperrrrrrr)r-ryr1rzrtr{rr3r|rXresidualself_attn_weightshidden_states_shapeoutputss r/r:zOPTDecoderLayer.forwards>!  $ E 55mDDM,:4> , '+%)+/) , ,  , , ( ( --mt|VZVc-dd  =0 ( E 55mDDM,1%--b-2D2DR2H2HII    $ A 11-@@M// **=99 //  --mt|VZVc-dd !M1778KLL ( A 11-@@M "  , )+ +Gr0r)NNNFFNN)r;r<r=r rr?r,rr7rr rr@rrr FloatTensorr:rArBs@r/rrsvvvyvXc]vvvvvv"_%0A6RRR2626+/,1$)3715PP|P!.P"%,/ P "% P $D> PD>Pu/0P!.P-.P u (51BEDU1U+V"WW XPPPSRPPPPPr0rcBeZdZUeed<dZdZdgZdZdZ dZ dZ dZ dZ dS)OPTPreTrainedModelr^modelTrcv|jj}t|tjrJ|jjd||j |jj dSdSt|tj rU|jjd||j +|jj|j  dSdSt|tj r?|jj d|jj dSdS)NrC)meanstdr)r^init_std isinstancerrnweightdatanormal_rbzero_ Embedding padding_idxrfill_)r-rDrs r/ _init_weightsz OPTPreTrainedModel._init_weights:s'k" fbi ( ( % M  & &CS & 9 9 9{&  &&((((('&  - - % M  & &CS & 9 9 9!- "6#56<<>>>>>.-  - - % M  $ $S ) ) ) K  " " $ $ $ $ $ % %r0N)r;r<r=r __annotations__base_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_attention_backend_supports_flash_attn_supports_sdpa_supports_flex_attn_can_compile_fullgraphrrr0r/rr-sc &*#*+"&N! % % % % %r0rceZdZdZdeffd Z ddeejdfdejdejd e d e f d Z e dejd e d e dejdejde f dZe ddeejdeejdeejd ee deejdee d ee dee dee deejdeejdeedeeeffdZxZS) OPTDecoderz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`OPTDecoderLayer`] Args: config: OPTConfig r^chtj|_j|_j|_j|_j|_tj jj |j|_ tjj|_j jkr'tjjj d|_nd|_j jkr'tjj jd|_nd|_jr-js&tjjj|_nd|_tjfdt3jD|_d|_|dS)NFrarc2g|]}t|S))r_)r).0ir^s r/ z'OPTDecoder.__init__..ps&$s$s$sa_Vq%I%I%I$s$s$sr0)r+r,rI layerdrop pad_token_idrmax_position_embeddingsmax_target_positions vocab_sizerrword_embed_proj_dim embed_tokensr$rcembed_positionsrn project_out project_inr_remove_final_layer_normrrr ModuleListrangenum_hidden_layerslayersgradient_checkpointing post_initr-r^r.s `r/r,zOPTDecoder.__init__Qs    ~ )!.$*$B! +L):F|jjd vr0|s.t j|j} tj| | } | S) Nflash_attention_2rCflex_attentionrFsdpa) inputs_embedsr2 is_trainingrrK)sequence_length target_lengthrMr| batch_size)cudaxpunpu)r^ranyrr7rr"get_seq_lengthis_compileabler_ignore_causal_mask_sdparPrMrget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positiondevicetypefinfomin_unmask_unattended) r-r1rr|rtr{past_seen_tokensusing_compilable_cacherMrr causal_mask min_dtypes r/_update_causal_maskzOPTDecoder._update_causal_maskws ; +/B B B)~/D.I.I.K.K)%%4 ; +/? ? ?.%,77 M!<^!L!L! ! @O?Z?99;;;`aCRC^!?!?di ; +v 5 5>T 5]n 5%>*'7 M   t"&,Q/ ! +??AAMMnel;;<$R((%7!; PP +')#)!, Q   K , 6 6*%*.DDD%E  E**.I0CKQZ[[Kr0rrrMrc ||dkr|}nMtj|j}tj||f|||j}|dkrtj|d}|tj||j|ddkz}|ddddddf |ddd}|| }|j d} |ddddddd| f|ddddddf |jz} | dk} |ddddddd| f | ||ddddddd| f<|S) aM Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerMrr)diagonalrrKr)r6r7rrfullrtriuarangerexpandclonerrV masked_fill) r1rrrMr|rrXrr mask_length padding_masks r/rz@OPTDecoder._prepare_4d_causal_attention_mask_with_cache_positions>  %.*<*<*>*>!*C*C(KK E**.I* -0Ye\j\qK!###jqAAA 5< n>STTTWeWmWmnprsWtWtt tK%dD!!!QQQ&67>>z1bRTUUK))//11 ,226 *111aaaL[L+@ANSTSTSTVZ\`bcbcbcScDdDgDg&EE   ,q0 5@AAAqqq,;,AV5W5c5c )66 AAAqqq!!!\k\12r0N input_ids head_maskrroutput_hidden_states return_dictr3rXr}c ||n |jj}||n |jj}||n |jj}| | n |jj} |du|duz rt d|jr%|jr|rt dd}|!| d|j d}|| |}|r|t|j}||nd} | *tj| | |j dz|j } |7| |j dz}tj|j d||j }|||| ||}| >tj|d } | |zdz } | dd| df} ||| | }|j||}|||jz}|rd nd}|rd nd}t3|gd gD]z\}}|s|dt7|jkrCt d|dt7|jd|dd{t;|jD]e\}}|r||fz }|jr tjg}||jkr4||f|| |||nd|||| d| }|d}|r ||dfz }f|j | |}|j!|!|}|r||fz }tE||||S)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(num_hidden_layers, num_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. for padding use -1. [What are position IDs?](../glossary#position-ids) cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length. Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FrK)r^rrrr5)r3rrzThe `z` should be specified for z layers, but it is for .)r1r3rzrtr{rr|last_hidden_statertry attentions)#r^r{rruse_return_dictrmrrPrirjrrrr rr7rronesrr8r9rrrVziprlenr enumeraterandrrrr)r-rr1rrtrrr{rr r3r|rXr seq_lengthr pos_embedsryall_hidden_statesall_self_attns attn_mask mask_nameidx decoder_layerdropout_probability layer_outputss r/r:zOPTDecoder.forwardsrN2C1N--TXT_Tq$8$D $+Jj "+!6IIDK??I   --i88M  ?0*$+>>>O?N?Z?99;;;`a  !"\ "2]5H5K"KTaThN  !)M,?,BBJ"Z (;A(> S`SghhhN.. M>?L]    <A>>>L(>9A=CCEEL'+;+<+<(<=L)).:JYe)ff ? & OOM::M% m6J(K(KK #7@BBD0:d%( k]$C$C   Iy$>>##A&3t{+;+;<<$3 33SEUEU33%NN,,Q/333 #,DK"8"8 6 6 C# 6!m%55!} &+jnn#&77)M *)3<3H3d /"3#-    M*!,M  6=#3"55  , 11-@@M   ' ,,];;M  2 -!1 1 &+++%     r0)F NNNNNNNNNNN)r;r<r=r>r r,rr7rr rr staticmethodr?rMrrrr@rrrrrr:rArBs@r/rrIsH#y######X#( BBelK78BlB B  B  BBBBH4 444{ 4  4  444\4l1515,0+/59$(,0/3&*3715t t E,-t !.t EL) t "% t   12 t D>t $D>t 'tnt d^t u/0t !.t -.t  u-- .t t t t t t t t r0rceZdZdeffd ZdZdZee dde e j de e j de e j d e e d e e jd e ed e ed e ede ede e j de e j deedeeeffdZxZS)OPTModelr^ct|t||_|dSr)r+r,rdecoderrrs r/r,zOPTModel.__init__s@    !&))  r0c|jjSrr$rr-s r/get_input_embeddingszOPTModel.get_input_embeddingss |((r0c||j_dSrr&r-rGs r/set_input_embeddingszOPTModel.set_input_embeddingss$) !!!r0Nrr1rrtrrr{rr r3r|rXr}c  ||n |jj}||n |jj}||n |jj}| | n |jj} |jd||| ||||||d| d | } t | j| j| j | j S)NT rr1r3rrtrrr{rr r|r r) r^r{rrrr$rr rtryr)r-rr1rrtrrr{rr r3r|rXdecoder_outputss r/r:zOPTModel.forwards"2C1N--TXT_Tq$8$D $+Jj "+!6IIDK+ $D>+ 'tn+ d^+ u/0+ !.+ -.+  u-- .+ + + ^+ + + + + r0r"c!eZdZdgZfdZdZdZdZdZe e dde e j d e e jd e e jd e ed e e jd e e j de ede ede ede ede e j de e jdeedeeeffdZxZS)OPTForCausalLMzlm_head.weightct|t||_t j|j|jd|_| dSNFra) r+r,r"rrrnrrlm_headrrs r/r,zOPTForCausalLM.__init__sc    f%% y!;V=NUZ[[[  r0c$|jjjSrrr$rr's r/r(z#OPTForCausalLM.get_input_embeddingsz!..r0c(||jj_dSrr5r*s r/r+z#OPTForCausalLM.set_input_embeddings*/ '''r0c||j_dSrrr$)r-r$s r/ set_decoderzOPTForCausalLM.set_decoders$ r0c|jjSrr:r's r/ get_decoderzOPTForCausalLM.get_decoders z!!r0Nrr1rrtrlabelsrr{rr r3r|rXr}c  ||n |jj}| | n |jj} | | n |jj} |jjd||| |||||| d| d | }||d}d}|5||j }|j ||fd|jj i| }t|||j |j|jS)an labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, OPTForCausalLM >>> model = OPTForCausalLM.from_pretrained("facebook/opt-350m") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious. I'm just a little bit of a weirdo." ```NTr-rrlosslogitsrtryrr)r^r{rrrr$r3rWrVr loss_functionrrrtryr)r-rr1rrtrr>rr{rr r3r|rXrrBrAs r/r:zOPTForCausalLM.forwardsBR2C1N--TXT_Tq$8$D $+Jj &1%P $D>P 'tnP d^P u/0P !.P +,P  u,, -P P P ^P P P P P r0r0a The OPT Model transformer with a sequence classification head on top (linear layer). [`OPTForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc`eZdZdeffd Ze ddeejdeej deej dee deej d eejd ee d ee d ee d ee deejde e effdZdZdZxZS)OPTForSequenceClassificationr^ct||j|_t||_t j|j|jd|_| dSr2) r+r, num_labelsr"rrrnrscorerrs r/r,z%OPTForSequenceClassification.__init__fsj     +f%% Yv94?QVWWW  r0Nrr1rrtrr>rr{rr r3r}c | | n |jj} ||||| ||||| |  } | d} || }||jdd\}}n|jdd\}}|jj|dkrt d|jjd}n|}||jjk|jtj }tj |jd|jtj }||z d}n)d}t|jjd |tj ||j |f}d}|Z|jjf|jdkr d |j_nN|jdkr7|jtjks|jtjkr d |j_n d |j_|jjd krWt-}|jdkr1|||}n|||}n|jjd krGt1}||d|j|d}n*|jjd krt5}|||}| s|f| ddz}||f|zn|St7||| j| j| jS)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N rtr1r3rrrr{rr rr)rz=Cannot handle batch sizes > 1 if no padding token is defined.rK)rrMz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r regressionsingle_label_classificationmulti_label_classificationr@)r^rrrKrrrmrVrr7int32rargmaxrirjr.r; problem_typerJrMr9r?r squeezerrrrrtryr)r-rr1rrtrr>rr{rr r3transformer_outputsryrBrrlast_non_pad_token non_pad_mask token_indices pooled_logitsrAloss_fctoutputs r/r:z$OPTForSequenceClassification.forwardosL*&1%!F!Fr!J!J  !#    >*ZZZ    u|Jv}MMMOaab   {'/?a''/;DK,,_q((flej.H.HFL\a\eLeLe/LDK,,/KDK,{'<77"99?a''#8M$9$9$;$;V^^=M=MNNDD#8M6::DD)-JJJ+--x 2 22t G GUWYY)-III,..x v66 F#%(;ABB(??F)-)9TGf$$v E/ /?-;*5     r0c$|jjjSrr5r's r/r(z1OPTForSequenceClassification.get_input_embeddingsr6r0c(||jj_dSrr5r*s r/r+z1OPTForSequenceClassification.set_input_embeddingsr8r0r)r;r<r=r r,rrr7r@rr rrrrr:r(r+rArBs@r/rHrHWsy156:15+/59-1$(,0/3&*37\ \ E,-\ !!23\ E-. \ "% \   12 \ )*\ D>\ $D>\ 'tn\ d^\ u/0\  u66 7\ \ \ ^\ |///0000000r0rHc|eZdZdeffd Ze ddeejdeej deej dee deej d eejd eejd ee d ee d ee dee deejde e effdZdZdZxZS)OPTForQuestionAnsweringr^ct|t||_t j|jd|_|dSr() r+r,r"rrrnr qa_outputsrrs r/r,z OPTForQuestionAnswering.__init__sX    f%% )F$>BB r0Nrr1rrtrstart_positions end_positionsrr{rr r3r}c | | n |jj} ||||| |||| | |  } | d}||}|dd\}}|d}|d}d}||t|dkr|d}t|dkr|d}|d}| d| |j }| d| |j }t|}|||}|||}||zdz }| s||f| ddz}||f|zn|St|||| j| j S) a Example: ```python >>> from transformers import AutoTokenizer, OPTForQuestionAnswering >>> import torch >>> torch.manual_seed(4) # doctest: +IGNORE_RESULT >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> # note: we are loading a OPTForQuestionAnswering from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random >>> model = OPTForQuestionAnswering.from_pretrained("facebook/opt-350m") >>> question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" >>> inputs = tokenizer(question, text, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> answer_offset = len(tokenizer(question)[0]) >>> predict_answer_tokens = inputs.input_ids[ ... 0, answer_offset + answer_start_index : answer_offset + answer_end_index + 1 ... ] >>> predicted = tokenizer.decode(predict_answer_tokens) >>> predicted ' a nice puppet' ```NrMrrrKr5) ignore_indexr))rA start_logits end_logitsryr)r^rrrasplitrTrWrrclamprVrrrryr)r-rr1rrtrrbrcrr{rr r3rUryrBrfrg total_loss ignored_indexrZ start_lossend_lossr[s r/r:zOPTForQuestionAnswering.forwards;`&1%_ $D>_ 'tn_ d^_ u/0_  u22 3_ _ _ ^_ B///0000000r0r_)r0r"rrHr_)rC)Br>typingrrrr7rtorch.nnrrr activationsr cache_utilsr r generationrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrrutils.deprecationrconfiguration_optr !torch.nn.attention.flex_attentionr!integrations.flex_attentionr" get_loggerr;rirr$Modulerfloatr[r]rrrr"r0rHr___all__rr0r/rsd,,,,,,,,,, AAAAAAAAAA!!!!!!........))))))>>>>>>BBBBBB999999 GFFFFFFF&&&&&&pppppppppppppp000000(((((( !!K;;;;;;JJJJJJ  H % %;;;;;BL;;;H%% I% <% % < % U\* %  %%%%%.])])])])])29])])])@ccccc0cccL%%%%%%%%6_ _ _ _ _ #_ _ _ D : : : : : !: : : zk k k k k 'k k k \    m0m0m0m0m0#5m0m0  m0`o0o0o0o0o00o0o0o0d   r0