`i+ d dl mZmZmZ ddlmZ ddlmZ ddlm Z e rd dl
Z
ddl m Z dd l
mZ d
Z G d d e ZdS )
)AnyUnionoverload )GenerationConfig)is_torch_available )PipelineN)%MODEL_FOR_TEXT_TO_SPECTROGRAM_MAPPING)SpeechT5HifiGanzmicrosoft/speecht5_hifiganc
h e Zd ZdZdZdZdZdZdZdZ e d Z
dddd fd
Z d Z d
Z
ed ed ed
eeef fd Zed ee d ed
eeeef fd Zd eeee f d
eeeef eeeef f f fd Z ddZd Z xZS )TextToAudioPipelinea
Text-to-audio generation pipeline using any `AutoModelForTextToWaveform` or `AutoModelForTextToSpectrogram`. This
pipeline generates an audio file from an input text and optional other conditional inputs.
Unless the model you're using explicitly sets these generation parameters in its configuration files
(`generation_config.json`), the following default values will be used:
- max_new_tokens: 256
Example:
```python
>>> from transformers import pipeline
>>> pipe = pipeline(model="suno/bark-small")
>>> output = pipe("Hey it's HuggingFace on the phone!")
>>> audio = output["audio"]
>>> sampling_rate = output["sampling_rate"]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
You can specify parameters passed to the model by using [`TextToAudioPipeline.__call__.forward_params`] or
[`TextToAudioPipeline.__call__.generate_kwargs`].
Example:
```python
>>> from transformers import pipeline
>>> music_generator = pipeline(task="text-to-audio", model="facebook/musicgen-small", framework="pt")
>>> # diversify the music generation by adding randomness with a high temperature and set a maximum music length
>>> generate_kwargs = {
... "do_sample": True,
... "temperature": 0.7,
... "max_new_tokens": 35,
... }
>>> outputs = music_generator("Techno music with high melodic riffs", generate_kwargs=generate_kwargs)
```
This pipeline can currently be loaded from [`pipeline`] using the following task identifiers: `"text-to-speech"` or
`"text-to-audio"`.
See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=text-to-speech).
TF )max_new_tokensN)vocoder
sampling_rate no_processorc d t j |i | || _ | j dk rt d d | _ | j j t j v r?|6t j t
| j j n|| _ || _ | j | j j j | _ | j | j j }| j j dd }|'| | dD ]M}t+ ||d }||| _ t+ |dd t+ |j |d }||| _ N| j 4| j s/t/ | j d r | j j j | _ d S d S d S d S )Ntfz5The TextToAudioPipeline is only available in PyTorch.generation_config) sample_rater codec_configfeature_extractor)super__init__r framework
ValueErrorr model __class__r valuesr from_pretrainedDEFAULT_VOCODER_IDtodevicer config__dict__getupdateto_dictgetattrr hasattr processorr )
selfr r r argskwargsr%
gen_configsampling_rate_namer s
x/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/pipelines/text_to_audio.pyr z TextToAudioPipeline.__init__a s $)&))) ) >T ! ! TUU U : #H#O#Q#Q Q Q ? /0BCCFFtzGXYYY
L + < #!% !4!BD % Z&F ,001DdKKJ%
j 0 0 2 2333&F
;
;" '0BD I I
,)6D&& V^T::F$+F,?ASUY$Z$ZM$0-:* %d.? %GDN\oDpDp %!% !A!OD & % % % % % c t |t r|g}| j j j dk r=| j j dd ddddd}| | |}| j r| j
n| j } ||fi |dd i}|S )
Nbarkmax_input_semantic_lengthr FT
max_length)r7 add_special_tokensreturn_attention_maskreturn_token_type_idspaddingreturn_tensorspt)
isinstancestrr r%
model_typer semantic_configr' r( r tokenizerr, )r- textr/
new_kwargs preprocessoroutputs r2
preprocessz TextToAudioPipeline.preprocess s dC 6D : '6 1 1 # 4 D H HIdfi j j&+)-).' J
f % % % F)-):Nt~~ dBBfBBTBBB
r3 c | || j }|d }|d }| j rT| || j }d|vr
| j |d< | | | j j di ||}nHt | r$t d| | j di ||d }| j
|
| }|S )N)r$ forward_paramsgenerate_kwargsr zYou're using the `TextToAudioPipeline` with a forward-only model, but `generate_kwargs` is non empty. For forward-only TTA models, please use `forward_params` instead of `generate_kwargs`. For reference, the `generate_kwargs` used here are: r ) _ensure_tensor_on_devicer$ r can_generater r( generatelenr keysr )r- model_inputsr/ rI rJ rF s r2 _forwardz TextToAudioPipeline._forward s; ..vdk.JJ 01 !23 : " " $ $ E " ; ;OTXT_ ; ` `O #/997;7M 34
! !/ 2 2 2(TZ(JJ<J>JJFF?##
dKZK_K_KaKad d
TZAA,A.AA!DF < #\\&))F
r3 text_inputsrI returnc d S NrK r- rS rI s r2 __call__z TextToAudioPipeline.__call__ s SVSVr3 c d S rV rK rW s r2 rX z TextToAudioPipeline.__call__ s _b_br3 c 8 t j |fi |S )a
Generates speech/audio from the inputs. See the [`TextToAudioPipeline`] documentation for more information.
Args:
text_inputs (`str` or `list[str]`):
The text(s) to generate.
forward_params (`dict`, *optional*):
Parameters passed to the model generation/forward method. `forward_params` are always passed to the
underlying model.
generate_kwargs (`dict`, *optional*):
The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a
complete overview of generate, check the [following
guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation). `generate_kwargs` are
only passed to the underlying model if the latter is a generative model.
Return:
A `dict` or a list of `dict`: The dictionaries have two keys:
- **audio** (`np.ndarray` of shape `(nb_channels, audio_length)`) -- The generated audio waveform.
- **sampling_rate** (`int`) -- The sampling rate of the generated audio waveform.
)r rX )r- rS rI r s r2 rX z TextToAudioPipeline.__call__ s$ 0 uww >>~>>>r3 c t | dd
| j |d< t | dd | j |d< | j |d< |r|ni |r|ni d}|i }i }|||fS )Nassistant_modelassistant_tokenizerrB )rI rJ )r* r\ rB r] )r- preprocess_paramsrI rJ paramspostprocess_paramss r2 _sanitize_parametersz(TextToAudioPipeline._sanitize_parameters s 4*D 1 1 =151EO - . 4. 5 5 A+/>OK (595MO 1 2 1? FnnB2A Ir
$ " &*<<z3TextToAudioPipeline.postprocess.. s7 #f#f#fWYBEEekE$J$J$P$P$R$R#f#f#fr3 rg rh r )r r% r@ r r> dicttupler, decodelistr# rj rk rl r )r- rd output_dict waveform_keyre s r2 postprocessz TextToAudioPipeline.postprocess s : '5 0 0"LL%L 4%&&
! .E5))
! 8 ~,,U33H eT " " X#f#f]e#f#f#fK #+;;e5;;#O#O#U#U#W#WK '+'9 O$r3 )NNN)__name__
__module__ __qualname____doc___load_processor_pipeline_calls_generate_load_image_processor_load_feature_extractor_load_tokenizerr _default_generation_configr rG rR r r? r rp rX rs r ra rv
__classcell__)r s @r2 r r s 2 2j O # O ! #O "2!1" " " '+$T (P (P (P (P (P (P (PT 0 B VCV3V4S>VVV XV
bDIbbdSVX[S[nI]bbb Xb? d3i 0? tCH~tDcN 33 4? ? ? ? ? ?8 = = = =. r3 r )typingr r r
generationr utilsr baser
rj models.auto.modeling_autor !models.speecht5.modeling_speecht5r r" r rK r3 r2 r s ( ' ' ' ' ' ' ' ' ' ) ) ) ) ) ) & & & & & & DLLLQQQQQQCCCCCC1 w w w w w( w w w w wr3