`iM(>ddlmZmZmZddlmZddlmZmZm Z m Z m Z m Z ddl mZmZe r ddlmZddlmZerdd lmZe r dd lZdd lmZe jeZeed d GddeZd S))AnyUnionoverload)GenerationConfig)add_end_docstringsis_tf_availableis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args)Image) load_image)'TF_MODEL_FOR_VISION_2_SEQ_MAPPING_NAMESN)$MODEL_FOR_VISION_2_SEQ_MAPPING_NAMEST) has_tokenizerhas_image_processorc jeZdZdZdZdZdZdZdZe dZ fdZ ddZ e d eed fd ed eeeeffd Ze d eeeed fd ed eeeeeffdZd eeeed ed fffd ZddZdZdZxZS)ImageToTextPipelinea Image To Text pipeline using a `AutoModelForVision2Seq`. This pipeline predicts a caption for a given image. Unless the model you're using explicitly sets these generation parameters in its configuration files (`generation_config.json`), the following default values will be used: - max_new_tokens: 256 Example: ```python >>> from transformers import pipeline >>> captioner = pipeline(model="ydshieh/vit-gpt2-coco-en") >>> captioner("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png") [{'generated_text': 'two birds are standing next to each other '}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This image to text pipeline can currently be loaded from pipeline() using the following task identifier: "image-to-text". See the list of available models on [huggingface.co/models](https://huggingface.co/models?pipeline_tag=image-to-text). TF)max_new_tokensctj|i|t|d||jdkrt nt dS)Nvisiontf)super__init__r check_model_type frameworkrr)selfargskwargs __class__s x/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/pipelines/image_to_text.pyrzImageToTextPipeline.__init__Tsg$)&)))$))) 7;~7M7M 3 3Sw     Nci}i}|||d<|||d<|||d<|*|d|vrtd|||j |j|d<|j|j|d<|j|d<||ifS)Nprompttimeoutrzp`max_new_tokens` is defined both as an argument and inside `generate_kwargs` argument, please use only 1 versionassistant_model tokenizerassistant_tokenizer) ValueErrorupdater+r-r,)r"rgenerate_kwargsr)r*forward_paramspreprocess_paramss r&_sanitize_parametersz(ImageToTextPipeline._sanitize_parameters[s  *0 h '  +2 i (  %/=N+ ,  &).>/.Q.Q &  ! !/ 2 2 2   +040DN, -  # /*..N; '484LN0 1 ."44r'inputsz Image.Imager$returnc dSNr"r4r$s r&__call__zImageToTextPipeline.__call__vsbeber'c dSr7r8r9s r&r:zImageToTextPipeline.__call__ystwtwr'c d|vr|d}|tdtj|fi|S)a Assign labels to the image(s) passed as inputs. Args: inputs (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`): The pipeline handles three types of images: - A string containing a HTTP(s) link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. max_new_tokens (`int`, *optional*): The amount of maximum tokens to generate. By default it will use `generate` default. generate_kwargs (`Dict`, *optional*): Pass it to send all of these arguments directly to `generate` allowing full control of this function. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A list or a list of list of `dict`: Each result comes as a dictionary with the following key: - **generated_text** (`str`) -- The generated text. imagesNzBCannot call the image-to-text pipeline without an inputs argument!)popr.rr:)r"r4r$r%s r&r:zImageToTextPipeline.__call__|sT< v  ZZ))F >abb buww11&111r'ct||}|tdt|ts t dt |d|jjj }|dkr| ||j }|j dkr| |j }||d j}|jjg|z}t#j|d }|d |in|d krC| |||j }|j dkr| |j }n|dkrs| ||j }|j dkr| |j }|||j }||nTt d|d| ||j }|j dkr| |j }|jjj dkr|d|d <|S)N)r*uPassing `prompt` to the `image-to-text` pipeline is deprecated and will be removed in version 4.48 of 🤗 Transformers. Use the `image-text-to-text` pipeline insteadz&Received an invalid text input, got - zy - but expected a single string. Note also that one single text can be provided for conditional image to text generation.git)r=return_tensorsptF)textadd_special_tokensr input_ids pix2struct)r= header_textrAzvision-encoder-decoder)rAz Model type z- does not support conditional text generation)rlogger warning_once isinstancestrr.typemodelconfig model_typeimage_processorr!todtyper,rE cls_token_idtorchtensor unsqueezer/)r"imager)r*rO model_inputsrE text_inputss r& preprocesszImageToTextPipeline.preprocesssv5'222     W   fc**  oT&\\ooo *5JU""#335QUQ_3`` >T))#/??4:#>#>L NN5NQQ[ !^89IE !L33==a@@ ##[)$<====|++#335feies3tt >T))#/??4:#>#>L777#335QUQ_3`` >T))#/??4:#>#>L"nnVDNnSS ##K0000!!hz!h!h!hiii //uT^/\\L~%%+tz:: :  '5 0 0V^(,L %r'c d|vr?t|dtr$td|dDrd|d<d|vr |j|d<||jj}|jj|fi||}|S)NrEc3K|]}|duV dSr7r8).0xs r& z/ImageToTextPipeline._forward..s&AA!AIAAAAAAr'generation_config)rJlistallr`r>rMmain_input_namegenerate)r"rXr0r4 model_outputss r&_forwardzImageToTextPipeline._forwards < ' '< 4d;; (AA|K'@AAAAA ()-L % o 5 5373IO/ 0 !!$*"<==+ +FVVlVoVV r'czg}|D]5}d|j|di}||6|S)Ngenerated_textT)skip_special_tokens)r,decodeappend)r"rerecords output_idsrecords r& postprocesszImageToTextPipeline.postprocesss[' # #J $."7"7(,#8##F NN6 " " " "r')NNNN)NN)__name__ __module__ __qualname____doc___pipeline_calls_generate_load_processor_load_image_processor_load_feature_extractor_load_tokenizerr_default_generation_configrr3rrrKrradictr:rZrfro __classcell__)r%s@r&rr.s4 $O #O!1!1"""     55556euS-%78eCeDQUVY[^V^Q_L`eeeXe wuT#Y]0C%CDwPSwX\]abfgjlogobp]qXrwwwXw"2uS$s)]DDW%WX"2"2"2"2"2"2H1111f,       r'r)typingrrr generationrutilsrr r r r r baserrPILr image_utilsrmodels.auto.modeling_tf_autorrTmodels.auto.modeling_autor get_loggerrprHrr8r'r&rs ('''''''''))))))54444444)((((((?WVVVVVVQLLLPPPPPP  H % %,,4UYZZZ[[DDDDD(DD\[DDDr'