`i_ddlZddlmZmZmZddlZddlZddlm Z ddlm Z ddl m Z ddl mZddlmZddlmZmZdd lmZdd lmZdd lmZmZmZdd lmZmZdd lm Z ddl!m"Z"m#Z#m$Z$ddl%m&Z&e#rddl'm(Z(e$j)e*Z+Gdde j,Z-Gdde j,Z.GddeZ/GddeZ0GddeZ1Gdde j,Z2Gdde j,Z3 dJd!e j,d"ej4d#ej4d$ej4d%eej4d&ee5d'e5d(eej4fd)Z6Gd*d+e j,Z7Gd,d-e j,Z8Gd.d/eZ9Gd0d1e j,Z:Gd2d3e j,Z;Gd4d5eZ<Gd6d7e j,Z=e"Gd8d9eZ> dKd:e?e@e@fd;e5dejBf d?ZCe"Gd@dAe>ZDdZEe"dBCGdDdEe>ZFe"dFCGdGdHe>ZGgdIZHdS)LN)CallableOptionalUnion)CrossEntropyLoss)ACT2FN)is_deepspeed_zero3_enabled)is_fsdp_managed_module)_prepare_4d_attention_mask#_prepare_4d_attention_mask_for_sdpa)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputCausalLMOutputSequenceClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)auto_docstringis_torch_flex_attn_availablelogging) HubertConfig)make_flex_block_causal_maskc$eZdZfdZdZxZS)HubertPositionalConvEmbeddingc&ttj|j|j|j|jdz|j|_d|_|j r tj |j|_nWtj j }ttj jdrtj jj }trddl}|j|jjd5||jdd|_dddn #1swxYwYt|jdr-|jjjj}|jjjj}n|jj}|jj}|j|||j||n||jdd|_t3|j|_t6|j|_dS) N) kernel_sizepaddinggroups weight_normr modifier_rankweight)namedimparametrizations)super__init__nnConv1d hidden_sizenum_conv_pos_embeddingsnum_conv_pos_embedding_groupsconv batch_normconv_pos_batch_norm BatchNorm1dutilsr"hasattrr(r deepspeedzeroGatheredParametersr% original0 original1weight_gweight_vregister_external_parameterHubertSamePadLayerr rfeat_extract_activation activation)selfconfigr"r6r;r< __class__s ~/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/hubert/modeling_hubert.pyr*z&HubertPositionalConvEmbedding.__init__3s' I    62a77       % I nV-?@@DOO(.Krx0-@@ D h7C )++ I    ^66ty7GWX6YYMM + DIH! L L LDIMMMMMMMMMMMMMMM49&8992#y9@JH#y9@JHH#y1H#y1H::4JJJ::4JJJJ'K aHHH )&*HII  !?@sD--D14D1c|dd}|j||}||}||}||}|dd}|S)Nrr) transposer1r0r r@rA hidden_statess rDforwardz%HubertPositionalConvEmbedding.forwardXs~%//155 ? & OOM::M -00  ]33  66 %//155 __name__ __module__ __qualname__r*rI __classcell__rCs@rDrr2sM#A#A#A#A#AJ       rJrc$eZdZfdZdZxZS)r>clt|dzdkrdnd|_dS)Nrrr)r)r*num_pad_remove)rAr.rCs rDr*zHubertSamePadLayer.__init__es: #:Q#>!#C#CaarJcJ|jdkr|ddddd|j f}|SNr)rSrGs rDrIzHubertSamePadLayer.forwardis;   " ")!!!QQQ0F43F2F0F*FGMrJrKrPs@rDr>r>dsLKKKKKrJr>c&eZdZdfd ZdZxZS)HubertNoLayerNormConvLayerrcZt|dkr|j|dz nd|_|j||_t j|j|j|j||j||j |_ t|j |_ dS)Nrrrstridebias)r)r*conv_dim in_conv_dim out_conv_dimr+r, conv_kernel conv_stride conv_biasr0rr?r@rArBlayer_idrCs rDr*z#HubertNoLayerNormConvLayer.__init__ps z1HubertFeatureEncoder.__init__..s>LLLGH*6AEBBBLLLrJrlayerc2g|]}t|S)r|)rhr~s rDrz1HubertFeatureEncoder.__init__..s'wwwA3FQGGGwwwrJz`config.feat_extract_norm` is z), but has to be one of ['group', 'layer']FT) r)r*feat_extract_normrqrangenum_feat_extract_layers ValueErrorr+ ModuleList conv_layersgradient_checkpointing_requires_grad)rArBrrCs ` rDr*zHubertFeatureEncoder.__init__s   #w . .3FQGGGHLLLLLQRXRpstRtLuLuLLLKK % 0 0wwwwQVW]WuQvQvwwwKKt1Ittt =55&+#"rJcP|D] }d|_ d|_dSNF) parameters requires_gradrrAparams rD_freeze_parametersz'HubertFeatureEncoder._freeze_parameterss4__&& ( (E"'E  #rJcr|dddf}|jr|jrd|_|jD] }||}|S)NT)rtrainingrr)rA input_valuesrH conv_layers rDrIzHubertFeatureEncoder.forwards[$QQQW-    /4= /*.M '* 6 6J&J}55MMrJ)rLrMrN__doc__r*rrIrOrPs@rDryrys\88#####"$$$       rJryc$eZdZfdZdZxZS)HubertFeatureProjectioncTt|j|_|jr+tj|jd|j|_tj|jd|j |_ tj |j |_ dS)Nroeps)r)r*feat_proj_layer_normr+rkr\layer_norm_epsrlLinearr- projectionDropoutfeat_proj_dropoutdropoutrArBrCs rDr*z HubertFeatureProjection.__init__s $*$?!  $ [ l6?2+>FDYZZZDO)FOB$79KLLz&":;; rJc|jr||}||}||}|Sre)rrlrrrGs rDrIzHubertFeatureProjection.forwardsF  $ ; OOM::M 66  ]33 rJrKrPs@rDrrsG<<<<<rJrmodulequerykeyvalueattention_maskscalingr head_maskc ||ddz}tj||dd|z} || |z} tj| d} || |ddddz} tj| ||j } tj| |} | dd } | | fS)Nrorrr'r)pr) sizetorchmatmulrFr+ functionalsoftmaxviewrr contiguous) rrrrrrrrkwargs attn_weights attn_outputs rDeager_attention_forwardrs**R..D(<s}}Q':':;;gEL!#n4 =((2(>>L#innQAq&A&AA =((6?([[L,|U33K''1--88::K  $$rJc>eZdZdZ ddededed ed ed ed eeffd Z dde j dee j dee j dee j deede e dee j ee j eee j ffdZxZS)HubertAttentionz=Multi-headed attention from 'Attention Is All You Need' paperrFTN embed_dim num_headsr is_decoderr[ is_causalrBc t||_||_||_||z|_||_|j|z|jkrtd|jd|d|jdz|_||_ ||_ tj ||||_ tj ||||_tj ||||_tj ||||_dS)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).r)r[)r)r*rrrhead_dimrBrrrrr+rk_projv_projq_projout_proj) rArrrrr[rrBrCs rDr*zHubertAttention.__init__s "" !Y.  MI %$. 8 83dn33%.333 }d* $"i 94@@@ i 94@@@ i 94@@@  )YTBBB rJrHkey_value_statesrlayer_head_maskoutput_attentionsrreturnc |du}|jdd\}} |r |jdn| } || d|jf} || d|jf} ||j| dd} |r|n|}||j| dd}||j| dd}t}|jj dkrt|jj }||| |||f|j sdn|j |j ||d|\}}||| d}||}||dfS)z#Input shape: Batch x Time x ChannelNrorreagerr)rrrr)shaperrrrFrrrrB_attn_implementationrrrrreshaperr)rArHrrrrris_cross_attentionbsztgt_lensrc_len q_input_shapekv_input_shape query_statescurrent_states key_states value_statesattention_interfacerrs rDrIzHubertAttention.forward/s.T9%*3B3/ W/AN"(++wgr4=9 wDM:7t{{=116 FPPQRTUVV -?R))]5T[[005~FPPQRTUVV 7t{{>227HRRSTVWXX (? ; +w 6 6"9$+:Z"[ $7$7      % $}>CC$,L/% % %  % % ! \"))#w;;FFHH mmK00 L$..rJ)rFTFN)NNNF)rLrMrNrintfloatboolrrr*rTensorrr tuplerIrOrPs@rDrr s^GG  )-CCCC C  C  CC&CCCCCCD481526,1 3/3/|3/#5<03/!. 3/ "%,/ 3/ $D> 3/-.3/ u|Xel3XeEL>Q5RR S3/3/3/3/3/3/3/3/rJrc$eZdZfdZdZxZS)HubertFeedForwardcttj|j|_tj|j|j|_ t|j trt|j |_n |j |_tj|j|j|_tj|j|_dSre)r)r*r+ractivation_dropoutintermediate_dropoutrr-intermediate_sizeintermediate_dense isinstance hidden_actstrrintermediate_act_fn output_densehidden_dropoutoutput_dropoutrs rDr*zHubertFeedForward.__init__fs $&Jv/H$I$I!"$)F,>@X"Y"Y f' - - 9'-f.?'@D $ $'-'8D $If&>@RSS j)>??rJc||}||}||}||}||}|Sre)rrrrrrGs rDrIzHubertFeedForward.forwardssg// >> 00?? 11-@@ ))-88 ++M:: rJrKrPs@rDrresL @ @ @ @ @rJrc&eZdZfdZddZxZS)HubertEncoderLayerctt|j|j|jd||_tj|j |_ tj |j|j |_ t||_tj |j|j |_dS)NFrrrrrBr)r)r*rr-num_attention_headsattention_dropout attentionr+rrrrkrrlr feed_forwardfinal_layer_normrs rDr*zHubertEncoderLayer.__init__~s ((0,    z&"788 ,v'9v?TUUU-f55 " V-?VEZ [ [ [rJNFc|}||||\}}}||}||z}||}|||z}||}|f}|r||fz }|SNrr)rrrlrrrArHrr attn_residualr_outputss rDrIzHubertEncoderLayer.forwards% )- .L]*8* * & |Q ]33 % 5  66 %(9(9-(H(HH --m<<  "  '   &GrJrrKrPs@rDrr}sQ \ \ \ \ \rJrc eZdZfdZ d dejdeejdeded ef d Z de ejdfd ejfd Z xZ S) HubertEncoderct|_t|_t jjj|_ t j j |_ t j fdtjD|_d|_dS)Nrc.g|]}tS)rrrrBs rDrz*HubertEncoder.__init__..s"$i$i$iA%7%?%?$i$i$irJFr)r*rBrpos_conv_embedr+rkr-rrlrrrrrnum_hidden_layerslayersrrs `rDr*zHubertEncoder.__init__s  ;FCC,v'9v?TUUUz&"788 m$i$i$i$ivOgIhIh$i$i$ijj &+###rJNFTrHrroutput_hidden_states return_dictc|rdnd}|rdnd}|;|ddd|jd}d||<|||}||} || z}||}||}tpt|} |j D]a} |r||fz}tj g} |j o| |j jk} | r| r| |||}|d}| rd}|r ||dfz}b|r||fz}|std|||fDSt!||| S) NrrorrrrNNc3K|]}||V dSrerrvs rD z(HubertEncoder.forward..(mmq_`_l_l_l_l_lmmrJlast_hidden_staterH attentions) unsqueezerepeatr_update_full_maskrrlrr r r rrandrrB layerdroprrrArHrrr r all_hidden_statesall_self_attentionsexpand_attention_maskposition_embeddings synced_gpusrdropout_probabilityskip_the_layer layer_outputss rDrIzHubertEncoder.forwards#7@BBD$5?bb4  %$2$<$BB gii  $.$/BB rJrHc||}||}||}||}|Sre)r7r8r:r;rGs rDrIzHubertAttnAdapterLayer.forward sL -00  m44  M22  m44 rJ)rLrMrNr*r FloatTensorrIrOrPs@rDr2r2s[ C C C C CU%6rJr2cXeZdZfdZ ddejdeejdefdZxZ S) !HubertEncoderLayerStableLayerNormctt|j|j|jd||_tj|j |_ tj |j|j |_ t||_tj |j|j |_t#|ddt%||_dSd|_dS)NFrrr4)r)r*rr-rrrr+rrrrkrrlrrrgetattrr2 adapter_layerrs rDr*z*HubertEncoderLayerStableLayerNorm.__init__s ((0,    z&"788 ,v'9v?TUUU-f55 " V-?VEZ [ [ [ 6-t 4 4 @!7!?!?D   !%D   rJNFrHrrcJ|}||}||||\}}}||}||z}||||z}|j|||z}|f}|r||fz }|Sr)rlrrrrrBrs rDrIz)HubertEncoderLayerStableLayerNorm.forward+s &  66 )- .L]*8* * & |Q ]33 % 5 %(9(9$:O:OP]:^:^(_(__   ))D,>,>},M,MMM "  '   &GrJr) rLrMrNr*rrrrrIrOrPs@rDr?r?s~&&&&&,26"' |!. rJr?cbeZdZfdZ d dZdeejdfdejfdZxZ S) HubertEncoderStableLayerNormct|_t|_t jjj|_ t j j |_ t j fdtjD|_d|_dS)Nrc.g|]}tSr)r?rs rDrz9HubertEncoderStableLayerNorm.__init__..Ms" ` ` `1 .v 6 6 ` ` `rJFrrs `rDr*z%HubertEncoderStableLayerNorm.__init__Fs  ;FCC,v'9v?TUUUz&"788 m ` ` ` `fF^@_@_ ` ` `   ',###rJNFTc|rdnd}|rdnd}|;|ddd|jd}d||<|||}||} || z}||}t pt|} |jD]a} |r||fz}tj g} |j o| |j j k} | r| r| |||}|d}| rd}|r ||dfz}b||}|r||fz}|std|||fDSt!||| S) Nrrorrrrrc3K|]}||V dSrerrs rDrz7HubertEncoderStableLayerNorm.forward..rrJr)rrrrrrr r r rrrrBrrlrrrs rDrIz$HubertEncoderStableLayerNorm.forwardQs#7@BBD$5?bb4  %$2$<$._conv_out_lengths&9\K7wWWWZ[[ [rJ)ziprBr_r`)rAr^rerrZs rD _get_feat_extract_output_lengthsz6HubertPreTrainedModel._get_feat_extract_output_lengthss\  \ \ \ $'t{'> @W#X#X Q Q K,,]KPPMMrJfeature_vector_lengthrc||dtj}|jd}tj||f|j|j}d|tj |jd|j|dz f<| dg d dg }|S)Nror)r,devicer)rj) rgsumtorlongrzerosr,rjarangeflipcumsumr)rArhroutput_lengths batch_sizes rD"_get_feature_vector_attention_maskz8HubertPreTrainedModel._get_feature_vector_attention_masks>>~?Q?QRT?U?UVVYYZ_Zdee#)!,  . /~7KTbTi   uv ^%9!%<^EZ[[[]kno]opq',,bT2299"==BBB4HHMMOOrJN)rLrMrNr__annotations__base_model_prefixmain_input_namesupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attnr]rr LongTensorrrgrtrrJrDrLrLs  $O&*#N[[[B eEDTVYDY>Z      ]b]m      rJrLr mask_prob mask_length min_masksrc@|\}dkrtdkrtdddtjdfd}|9|dnfd t|D}tj |ft }g} |} | d kr|S|D]} || } tj tj | dz z | d } t| d krdz }n| d }tj| tj| | z tj |zg} | | tj| } tj| dddddf|| f} | || z} tj ddddf}tj||| f|| z}| |z} | dz krdz | | dz k<tj|| dd|S)an Computes random mask spans for a given shape. Used to implement [SpecAugment: A Simple Data Augmentation Method for ASR](https://huggingface.co/papers/1904.08779). Note that this method is not optimized to run on TPU and should be run on CPU as part of the preprocessing during training. Args: shape: The shape for which to compute masks. This should be of a tuple of size 2 where the first element is the batch size and the second element is the length of the axis to span. mask_prob: The percentage of the whole axis (between 0 and 1) which will be masked. The number of independently generated mask spans of length `mask_length` is computed by `mask_prob*shape[1]/mask_length`. Note that due to overlaps, `mask_prob` is an upper bound and the actual percentage will be smaller. mask_length: size of the mask min_masks: minimum number of masked spans attention_mask: A (right-padded) attention mask which independently shortens the feature axis of each batch dimension. rz&`mask_length` has to be bigger than 0.zO`mask_length` has to be smaller than `sequence_length`, but got `mask_length`: z and `sequence_length`: `ct|zz z}t|}|zkrz}|dz z |krt|dz z d}|S)z;Given input length, compute how many spans should be maskedrr)rmax)rdnum_masked_spanepsilonr~r}rsequence_lengths rDcompute_num_masked_spanz6_compute_mask_indices..compute_num_masked_spans~i,6DwNOOoy99 [ (? : :-.&s 9 9 9!o 9 9 9rJr,rF)replace)rnprandomritemdetachrktolistrrnrchoicerolen concatenateonesint32appendarray broadcast_torrput_along_axis)rr}r~rrrsrr^ spec_aug_maskspec_aug_mask_idxsmax_num_masked_spanrdrspec_aug_mask_idxdummy_mask_idxoffsetsrrs `` ` @@rD_compute_mask_indicesrsP0#(JQABBB_$$ :^i : :'6 : : :   innQ$$&&G         $  % ##B''..000 9 9 9 9uZ'8'8 9 9 9Hj/:$GGGM11/BBa%55 11,??I,, IlkAo6 7 7RW-    ! !Q & &-q0NN.q1NN (;o(MUWU] ^ ^ ^ao o p   !!"34444"455111aaa:&5H+(V,33J@SVa@abbi $$T4]3Gog 4G'UVV^^'+5G,g5/A"555GVYZGZ-!0CCDm%7B??? rJc*eZdZdeffd Z ddejdeejdeejfdZ e ddeej deej deejd ee d ee d ee d e eeffd ZxZS)rZrBct|||_t||_t ||_|jdks |jdkrBtj tj |j |_|jrt#||_nt'||_|dS)Nr)r)r*rBryfeature_extractorrfeature_projectionmask_time_probmask_feature_probr+ Parameterrrr-r[rQdo_stable_layer_normrEencoderr post_initrs rDr*zHubertModel.__init__fs     !5f!=!="9&"A"A  3 & &&*BS*H*H%'\%,v?Q2R2R2[2[2]2]%^%^D "  & 17??DLL(00DL rJNrHmask_time_indicesrct|jdds|S|\}}}|#|j|j||<n|jjdkr|jr|t||f|jj|jj ||jj }tj ||j tj}|j|j||<|jjdkr|jrt||f|jj|jj|jj}tj ||j tj}|dddfd|d}d||<|S) z Masks extracted features along time axis and/or along feature axis according to [SpecAugment](https://huggingface.co/papers/1904.08779). apply_spec_augmentTNr)r}r~rr)rjr,)r}r~rro)rArBrrQrlr,rrrmask_time_lengthmask_time_min_masksrr0rjrrmask_feature_lengthmask_feature_min_masksexpand)rArHrrrsrr-mask_feature_indicess rD_mask_hidden_stateszHubertModel._mask_hidden_statesxst{$8$?? ! 4A3E3E3G3G0 O[  (/3/E/H/HI\/]/]M+ , , [ '! + + + 5_-+4 K8-+9 !!! !& ->}G[chcm n n n /3/E/H/HI\/]/]M+ , ; (1 , , ,#8[)+7 K;+< $$$ $)<0D]Mainis#t#t#t #74#@#G#GO]_#`#` 23M. /rJrrr r rc||n |jj}||n |jj}||n |jj}||}|dd}|!||jd|}||}| ||}| |||||} | d}|s|f| ddzSt|| j | j S)a1 mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. Example: ```python >>> from transformers import AutoProcessor, HubertModel >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft") >>> model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft") >>> def map_to_array(example): ... example["speech"] = example["audio"]["array"] ... return example >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.map(map_to_array) >>> input_values = processor(ds["speech"][0], return_tensors="pt").input_values # Batch size 1 >>> hidden_states = model(input_values).last_hidden_state ```Nrr)rrrr r rr)rBrr use_return_dictrrFrtrrrrrrHr) rArrrrr r extract_featuresrHencoder_outputss rDrIzHubertModel.forwards@H2C1N--TXT_Tq$8$D $+Jj &1%59 ,,(,$E$56,!!12 ,,,,\269=,0/3&*D D u|,D !.D $E$56 D $D> D 'tn D d^D  uo% &D D D ^D D D D D rJrZzn Hubert Model with a `language modeling` head on top for Connectionist Temporal Classification (CTC). ) custom_introceZdZddeeffd ZdZdZdZdZ e ddee j d ee j d ee d ee d ee d ee j deeeffdZxZS) HubertForCTCN target_langct|t||_t j|j|_||_|j td|j dt|dr|j r|jn|j}t j||j |_|dS)a0 target_lang (`str`, *optional*): Language id of adapter weights. Adapter weights are stored in the format adapter..safetensors or adapter..bin. Only relevant when using an instance of [`HubertForCTC`] with adapters. Uses 'eng' by default. NzYou are trying to instantiate z with a configuration that does not define the vocabulary size of the language model head. Please instantiate the model as follows: `HubertForCTC.from_pretrained(..., vocab_size=vocab_size)`. or define `vocab_size` of your model's configuration. add_adapter)r)r*rZrMr+r final_dropoutrr vocab_sizerrCr5routput_hidden_sizer-rlm_headr)rArBrrrCs rDr*zHubertForCTC.__init__s    !&)) z&"677 &   $HHHH *1)G)G vFL^ vF % %djdv y!3V5FGG  rJc|j}|)t|jddtd|d|2t|jddtddS|||ddSdS)a' This method overwrites [`~PreTrainedModel.tie_weights`] so that adapter weights can be correctly loaded when passing `target_lang=...` to `from_pretrained(...)`. This method is **not** supposed to be called by the user and is prone to be changed in the future. Nr4zCannot pass `target_lang`: z- if `config.adapter_attn_dim` is not defined.z)By default `target_lang` is set to 'eng'.T) force_load)rrArBrloggerinfo load_adapter)rArs rD tie_weightszHubertForCTC.tie_weightss&  "wt{||jjkrt d|jj||||||}|d}||}||} d} |Z||ntj |tj }| | d tj } |dk} | d} || }tj| dtjdd}tjjd 5tj||| | |jj|jj|jj } dddn #1swxYwY|s| f|t6dz}| | f|zn|St9| | |j|j S) a labels (`torch.LongTensor` of shape `(batch_size, target_length)`, *optional*): Labels for connectionist temporal classification. Note that `target_length` has to be smaller or equal to the sequence length of the output logits. Indices are selected in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size - 1]`. Nz$Label values must be <= vocab_size: rrrro)r'r,rF)enabled)blank reduction zero_infinitylosslogitsrHr)rBrrrrrMrrr ones_likermrgrkrl masked_selectr+r log_softmaxfloat32rFbackendscudnnflagsctc_loss pad_token_idctc_loss_reductionctc_zero_infinity_HIDDEN_STATES_START_POSITIONrrHr)rArrrr r rrrHrrr^ labels_masktarget_lengthsflattened_targets log_probsoutputs rDrIzHubertForCTC.forwardDs"&1% HSM:<<<* & & &;;;(((26,0/3&*)-D D u|,D !.D $D> D 'tn D d^ D &D  un$ %D D D ^D D D D D rJrz Hubert Model with a sequence classification head on top (a linear layer over the pooled output) for tasks like SUPERB Keyword Spotting. ceZdZfdZdZdZdZe ddee j dee j dee d ee d ee d ee j d e e effd ZxZS)r\ct|t|dr|jrt dt ||_|jdz}|jr.tj tj ||z |_ tj|j|j|_tj|j|j|_|dS)Nrz]Sequence classification does not support the use of Hubert adapters (config.add_adapter=True)r)r)r*r5rrrZrMr use_weighted_layer_sumr+rrrrRrr-classifier_proj_size projector num_labels classifierr)rArB num_layersrCs rDr*z(HubertForSequenceClassification.__init__s    6= ) ) f.@ o "&)) -1  ( S!#ej.D.Dz.Q!R!RD 6#5v7RSS)F$?ARSS rJcbtjdt|dS)z Calling this function will disable the gradient computation for the feature encoder so that its parameters will not be updated during training. rNrrs rDrz8HubertForSequenceClassification.freeze_feature_extractorrrJcB|jjdSrrrs rDrz6HubertForSequenceClassification.freeze_feature_encoderrrJcL|jD] }d|_ dSrrrs rDrz1HubertForSequenceClassification.freeze_base_modelrrJNrrrr r rrcd||n |jj}|jjrdn|}||||||}|jjrx|t}t j|d}tj |j d} || dddz d}n|d}| |}||d} n||jd|} | ddd|jd} d || <| d| d ddz } || } d}|Kt)}|| d|jj| d}|s| f|tdz}||f|zn|St-|| |j|j S) a  input_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`): Float values of input raw speech waveform. Values can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoProcessor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. See [`HubertProcessor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NTrrrrorrrr)rBrrrMrrstackr+rrrRrrkrrOrtrrrrrrrrHr)rArrrr r rrrH norm_weights pooled_output padding_maskexpand_padding_maskrrloss_fctrs rDrIz'HubertForSequenceClassification.forwardsW.&1%->r1a-H-HHMMRSMTTMM#AJM}55  !)..1.55MMBB=CVWXCY[ijjL"."8"8"<"<"C"CAq-J]^_J`"a"a 25M.. /)--!-44|7G7GA7G7N7N7S7STVXY7Z7ZZM//  '))H8FKKDK,BCCV[[QS__UUD FY)F)G)G!HHF)-)9TGf$$v E'!/)     rJr)rLrMrNr*rrrrrrrrrrrrIrOrPs@rDr\r\s" & & &;;;(((26,0/3&*)-B B u|,B !.B $D> B 'tn B d^ B &B  u.. /B B B ^B B B B B rJr\)rr\rZrL)NrNrU)Irtypingrrrnumpyrrtorch.nnr+r activationsrintegrations.deepspeedr integrations.fsdpr modeling_attn_mask_utilsr r modeling_flash_attention_utilsr modeling_layersrmodeling_outputsrrrmodeling_utilsrrprocessing_utilsrr4rrrconfiguration_hubertrintegrations.flex_attentionr get_loggerrLrModulerr>rWrhrqryrrrrrrrrr2r?rErLrrr|ndarrayrrZrrr\__all__rrJrDrsY,,,,,,,,,,, %%%%%%!!!!!!@@@@@@777777ggggggggBBBBBB999999YYYYYYYYYYFFFFFFFF&&&&&&JJJJJJJJJJ...... !!KJJJJJJ  H % %/////BI///d!;*9690#####29###Lbi0 $(,%% I% <% % < % U\* % e_ %% %%%%%