`iEdZddlZddlmZmZmZddlZddlm Z m Z m Z m Z ddl mZddlmZddlmZmZmZejeZGd d eZd gZdS) z%Feature extractor class for SpeechT5.N)AnyOptionalUnion)mel_filter_bankoptimal_fft_length spectrogramwindow_function)SequenceFeatureExtractor) BatchFeature)PaddingStrategy TensorTypeloggingceZdZdZddgZ d1dedededededededededededededeffd Z e d2de e j de e j ded e e j fd!Zd"e j d e j fd#Z d3d%eee j e ee e j e e efd&eee j e ee e j e e efd'eeeefd(eed)ed*eedeed+eeeefdeed efd,Z d4d-ee j e ee e j e e efd.ed'eeeefd(eed)ed*eedeed+eeeefd efd/Zd eeefffd0 ZxZS)5SpeechT5FeatureExtractora Constructs a SpeechT5 feature extractor. This class can pre-process a raw speech signal by (optionally) normalizing to zero-mean unit-variance, for use by the SpeechT5 speech encoder prenet. This class can also extract log-mel filter bank features from raw speech, for use by the SpeechT5 speech decoder prenet. This feature extractor inherits from [`~feature_extraction_sequence_utils.SequenceFeatureExtractor`] which contains most of the main methods. Users should refer to this superclass for more information regarding those methods. Args: feature_size (`int`, *optional*, defaults to 1): The feature dimension of the extracted features. sampling_rate (`int`, *optional*, defaults to 16000): The sampling rate at which the audio files should be digitalized expressed in hertz (Hz). padding_value (`float`, *optional*, defaults to 0.0): The value that is used to fill the padding values. do_normalize (`bool`, *optional*, defaults to `False`): Whether or not to zero-mean unit-variance normalize the input. Normalizing can help to significantly improve the performance for some models. num_mel_bins (`int`, *optional*, defaults to 80): The number of mel-frequency bins in the extracted spectrogram features. hop_length (`int`, *optional*, defaults to 16): Number of ms between windows. Otherwise referred to as "shift" in many papers. win_length (`int`, *optional*, defaults to 64): Number of ms per window. win_function (`str`, *optional*, defaults to `"hann_window"`): Name for the window function used for windowing, must be accessible via `torch.{win_function}` frame_signal_scale (`float`, *optional*, defaults to 1.0): Constant multiplied in creating the frames before applying DFT. This argument is deprecated. fmin (`float`, *optional*, defaults to 80): Minimum mel frequency in Hz. fmax (`float`, *optional*, defaults to 7600): Maximum mel frequency in Hz. mel_floor (`float`, *optional*, defaults to 1e-10): Minimum value of mel frequency banks. reduction_factor (`int`, *optional*, defaults to 2): Spectrogram length reduction factor. This argument is deprecated. return_attention_mask (`bool`, *optional*, defaults to `True`): Whether or not [`~SpeechT5FeatureExtractor.__call__`] should return `attention_mask`. input_valuesattention_mask>FP@ hann_window?绽|=T feature_size sampling_rate padding_value do_normalize num_mel_bins hop_length win_length win_functionframe_signal_scalefminfmax mel_floorreduction_factorreturn_attention_maskc tjd |||d|||_||_||_||_||_||_| |_| |_ | |_ | |_ | |_ ||zdz|_ ||zdz|_t|j |_|jdzdz|_t%|j |jd|_t)|j|j|j |j |jdd|_| d krt/jd t2| d krt/jd t2dSdS)N)rr r!irrT) window_lengthnameperiodicslaney)num_frequency_binsnum_mel_filters min_frequency max_frequencyr norm mel_scalerzeThe argument `frame_signal_scale` is deprecated and will be removed in version 4.30.0 of Transformersg@zcThe argument `reduction_factor` is deprecated and will be removed in version 4.30.0 of Transformers)super__init__r"r,r#r$r%r&r'r(r)r*r+ sample_size sample_stridern_fftn_freqsr windowrr mel_filterswarningswarn FutureWarning)selfrr r!r"r#r$r%r&r'r(r)r*r+r,kwargs __class__s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/speecht5/feature_extraction_speecht5.pyr:z!SpeechT5FeatureExtractor.__init__Ns}$ wl-_lwwpvwww(%:"($$("4  " 0% 5='-74?'(899  a1, %D4D4K\gklll *#| -)),     $ $ Mw    s " " Mu      # "returnc|tj|tj}g}t||dD]\}}||d|z tj|d|dzz }||jdkr|||d<| |n d|D}|S)z[ Every array in the list is normalized to have zero mean and unit variance NHz>rcg|]C}||z tj|dzz DS)rL)meannpsqrtvar).0xs rG zDSpeechT5FeatureExtractor.zero_mean_unit_var_norm..s@"b"b"bPQALBGAEEGGdN4K4K#K"b"b"brH) rOarrayint32zipsumrNrPrQshapeappend)rrr!normed_input_valuesvectorlength normed_slices rGzero_mean_unit_var_normz0SpeechT5FeatureExtractor.zero_mean_unit_var_norms  %Xnbh??N"$ "%lN4F4Fr4J4J"K"K 9 9 &)=)=)?)? ?276RYSYRY?K^K^K`K`cgKgChChh L.q111,9L)#**<8888  9#c"bUa"b"b"b ""rH one_waveformc zt||j|j|j|j|j|jd}|jS)zZ Extracts log-mel filterbank features for one waveform array (unbatched). log10)r? frame_lengthr$ fft_lengthr@r*log_mel)r r?r;r<r=r@r*T)rDr` log_mel_specs rG_extract_mel_featuresz.SpeechT5FeatureExtractor._extract_mel_featuressI# ;))z(n    ~rHNaudio audio_targetpadding max_length truncationpad_to_multiple_ofreturn_tensorsc ||td| 2| |jkr&td|d|jd|jd| d n(td|jjd ||j|d ||||||fi| } nd} |@|j|d ||||||fi| } | | S| d | d <| d} | | | d<| S)aA Main method to featurize and prepare for the model one or several sequence(s). Pass in a value for `audio` to extract waveform features. Pass in a value for `audio_target` to extract log-mel spectrogram features. Args: audio (`np.ndarray`, `list[float]`, `list[np.ndarray]`, `list[list[float]]`, *optional*): The sequence or batch of sequences to be processed. Each sequence can be a numpy array, a list of float values, a list of numpy arrays or a list of list of float values. This outputs waveform features. Must be mono channel audio, not stereo, i.e. single float per timestep. audio_target (`np.ndarray`, `list[float]`, `list[np.ndarray]`, `list[list[float]]`, *optional*): The sequence or batch of sequences to be processed as targets. Each sequence can be a numpy array, a list of float values, a list of numpy arrays or a list of list of float values. This outputs log-mel spectrogram features. padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `False`): Select a strategy to pad the returned sequences (according to the model's padding side and padding index) among: - `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single sequence if provided). - `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum acceptable input length for the model if that argument is not provided. - `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different lengths). max_length (`int`, *optional*): Maximum length of the returned list and optionally padding length (see above). truncation (`bool`): Activates truncation to cut input sequences longer than *max_length* to *max_length*. pad_to_multiple_of (`int`, *optional*): If set will pad the sequence to a multiple of the provided value. This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. return_attention_mask (`bool`, *optional*): Whether to return the attention mask. If left to the default, will return the attention mask according to the specific feature_extractor's default. [What are attention masks?](../glossary#attention-mask) return_tensors (`str` or [`~utils.TensorType`], *optional*): If set, will return tensors instead of list of python integers. Acceptable values are: - `'tf'`: Return TensorFlow `tf.constant` objects. - `'pt'`: Return PyTorch `torch.Tensor` objects. - `'np'`: Return Numpy `np.ndarray` objects. sampling_rate (`int`, *optional*): The sampling rate at which the `audio` or `audio_target` input was sampled. It is strongly recommended to pass `sampling_rate` at the forward call to prevent silent errors. Nz9You must provide either `audio` or `audio_target` values.z3The model corresponding to this feature extractor: z& was trained using a sampling rate of zB. Please make sure that the provided audio input was sampled with z and not .zDIt is strongly recommended to pass the `sampling_rate` argument to `zN()`. Failing to do so can result in silent errors that might be hard to debug.FTrlabelsrdecoder_attention_mask) ValueErrorr loggerwarningrF__name___process_audioget)rDrirjrkrlrmrnr,ror rEinputs inputs_targetrss rG__call__z!SpeechT5FeatureExtractor.__call__s~ =\1XYY Y  $ 222 F$FF*FF*FF5BFFF3 NN\W[WeWn\\\     (T("%    FFF  #/D/"%    M~$$#0#@x )6):):;K)L)L&)57MF34 rHspeech is_targetc t|tjot|jdk} | r*t|jdkrt d| pHt|t tfo,t|dtjtt f} | r d|D}n| s;t|tjs!tj|tj }n^t|tjrD|j tj tj ur| tj }| s|g}j } |r,fd|D} td| i}j_ ntd|i}j|f|||||d | }| _ |d}t|dtjsd |D|d<nt|tjs[t|dtjr;|dj tj tj urd |D|d<nat|tjrG|j tj tj ur"| tj |d<|d }|d |D|d <|sTjrM||t(jur|nd}|d|j|d<|||}|S)Nrrz2Only mono-channel audio is supported for input to rcNg|]"}tj|tj#SdtyperOasarrayfloat32)rRr}s rGrTz;SpeechT5FeatureExtractor._process_audio..=s)PPPvbjrz:::PPPrHrc:g|]}|Sr8)rh)rRwaveformrDs rGrTz;SpeechT5FeatureExtractor._process_audio..Ls'TTT228<<TTTrHr)rkrlrmrnr,cNg|]"}tj|tj#SrrrRrUs rGrTz;SpeechT5FeatureExtractor._process_audio..as*,k,k,kUZRZRZ-P-P-P,k,k,krHcLg|]!}|tj"Sr8)astyperOrrs rGrTz;SpeechT5FeatureExtractor._process_audio..gs&,`,`,`%U\\"*-E-E,`,`,`rHrcNg|]"}tj|tj#Sr)rOrrVrs rGrTz;SpeechT5FeatureExtractor._process_audio..ns*.m.m.mUZrz%rx/P/P/P.m.m.mrH)rl)rr!) isinstancerOndarraylenrYrtlisttuplerrrfloat64rrr r#padryr"_get_padding_strategiesr DO_NOT_PADr_r!convert_to_tensors)rDr}r~rkrlrmrnr,rorEis_batched_numpy is_batchedfeature_size_hackfeaturesencoded_inputs padded_inputsrrs` rGrxz'SpeechT5FeatureExtractor._process_audio)s&fbj99Sc&,>O>ORS>S  ZFL 1 1A 5 5XRVXXYY Y% ve} - - d:fQi"*V[]aIb3c3c   /PPPPPFF /Jvrz$B$B /Zbj999FF  + + / @T@T0T0T]]2:..F XF!-  DTTTTVTTTH)>8*DEEN $ 1D  )>6*BCCN   !!1"7     .%^4 ,q/2:66 L,k,k^j,k,k,kM. ) )<44 L<?BJ77 LQ%"*)=)===,`,`S_,`,`,`M. ) )  bj 1 1 Ll6HBHUWU_L`L`6`6`,8,?,? ,K,KM. )'**+;<<  %.m.m^l.m.m.mM* + T. //J/OOWfWqqq  -1,H,Hn-n\`\n-I--M. )  %)<<^LLMrHcht}gd}|D] }||vr||= |S)N)r?r@r;r<r=r>)r9to_dict)rDoutputnamesr/rFs rGrz SpeechT5FeatureExtractor.to_dictsI""^]] ! !Dv~~4L rH)rrrFrrrrrrrrrT)r) NNFNFNNNN)FFNFNNN)rw __module__ __qualname____doc__model_input_namesintfloatboolstrr: staticmethodrrOrr_rhrrr rr r|rxdictrr __classcell__)rFs@rGrrs**X()9:""")$' !&*:::: :  :  ::::"::::: $::::::xbe##2:&#8NPTUYZ_U`Pa%abcstS/12 s SM s  s%SMs (~s!sJ!78s }s ssssp 5:$( ,004;?UUbj$u+tBJ/?d5kARRSUUtS/12 U SM U  U%SMU (~U!sJ!78U UUUUn c3h          rHr)rrAtypingrrrnumpyrO audio_utilsrrr r !feature_extraction_sequence_utilsr feature_extraction_utilsr utilsr rr get_loggerrwrur__all__r8rHrGrs,+''''''''''\\\\\\\\\\\\IIIIII4444449999999999  H % %jjjjj7jjjZ & &rH