`i"dZddlZddlmZmZddlmZddlmZddl m Z m Z m Z ddl mZmZmZdd lmZd d lmZejeZGd d e dZGdde ZdgZdS)zq Processor class for InstructBLIP. Largely copy of Blip2Processor with addition of a tokenizer for the Q-Former. N)OptionalUnion) BatchFeature) ImageInput)ProcessingKwargsProcessorMixinUnpack) AddedTokenPreTokenizedInput TextInput)logging) AutoTokenizerc ,eZdZdddddddddd idZdS)InstructBlipProcessorKwargsTFr) add_special_tokenspaddingstridereturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mappingreturn_token_type_ids return_lengthverbose) text_kwargs images_kwargsN)__name__ __module__ __qualname__ _defaults/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/instructblip/processing_instructblip.pyrr!sF#').*/&+%*"     IIIr#rF)totalc eZdZdZgdZdZdZdZdfd Z dde e de e e ee ee fd eed efd Zed Zfd ZefdZxZS)InstructBlipProcessora Constructs an InstructBLIP processor which wraps a BLIP image processor and a LLaMa/T5 tokenizer into a single processor. [`InstructBlipProcessor`] offers all the functionalities of [`BlipImageProcessor`] and [`AutoTokenizer`]. See the docstring of [`~BlipProcessor.__call__`] and [`~BlipProcessor.decode`] for more information. Args: image_processor (`BlipImageProcessor`): An instance of [`BlipImageProcessor`]. The image processor is a required input. tokenizer (`AutoTokenizer`): An instance of ['PreTrainedTokenizer`]. The tokenizer is a required input. qformer_tokenizer (`AutoTokenizer`): An instance of ['PreTrainedTokenizer`]. The Q-Former tokenizer is a required input. num_query_tokens (`int`, *optional*):" Number of tokens used by the Qformer as queries, should be same as in model's config. )image_processor tokenizerqformer_tokenizer)BlipImageProcessorBlipImageProcessorFastrNc t|ds5tddd|_||jgdn |j|_||_t |||dS)N image_tokenzFT) normalizedspecial)special_tokens)hasattrr r. add_tokensnum_query_tokenssuper__init__)selfr(r)r*r4kwargs __class__s r$r6zInstructBlipProcessor.__init__Jsy-00 5))tTTTD   $"2!3D I I I I(4D  0 )5FGGGGGr#imagestextr8returnc  ||td|jtfd|jji|}|ddd}i}|ct |tr|g}n?t |ts*t |dtstd|j |fi|d} | d|d <| d |d <|d d |dd xx|j zcc<|j|fi|d} |f|j j |j z} d |dd<d |dd<d |dd<|j| fi|d | D] fd| D| <|| |)|j|fi|d} || t!||}|S)a This method uses [`BlipImageProcessor.__call__`] method to prepare image(s) for the model, and [`BertTokenizerFast.__call__`] to prepare text for the model. Please refer to the docstring of the above two methods for more information. Args: images (`ImageInput`): The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch tensor. Both channels-first and channels-last formats are supported. text (`TextInput`, `PreTokenizedInput`, `list[TextInput]`, `list[PreTokenizedInput]`): The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). Nz,You have to specify at least images or text.tokenizer_init_kwargsrreturn_tensorsrzAInvalid input text. Please provide a string, or a list of strings input_idsqformer_input_idsattention_maskqformer_attention_mask max_lengthFrr truncationc&g|] }|zSr"r").0sampleimage_text_encodingks r$ z2InstructBlipProcessor.__call__..s$'g'g'gF(;A(>(G'g'g'gr#r) tensor_type) ValueError _merge_kwargsrr) init_kwargspop isinstancestrlistr*getr4r.contentupdater(r)r7r:r;audiovideosr8 output_kwargsr?encodingqformer_text_encoding text_encoding image_tokensimage_encodingrIrJs @@r$__call__zInstructBlipProcessor.__call__Ts, >dlKLL L** '  "&."<   '}599:JDQQ  $$$ fvd++ fJtAw4L4L f !deee$:D$:4$`$`=Q^C_$`$` !,A,E,Ek,R,RH( )1F1J1JK[1\1\H- .]+// ==Im,\:::d>SS:::*DN4PP=3OPPM!#/7$:OO EJ m,-AB:? m,Y7=B m,\:&4dn\&b&b]S`Ea&b&b#&hhA'g'g'g'g'gVcdeVf'g'g'gM!$$ OOM * * *  1T1&[[M/>. ) ) db~bbbcc c NT2222!#n>Q!R!R ../EFFF.@  8 O " "#6 7 7 7)%'').CCFCC  5 OO 34 4OOr#c tj|fi|}t|tr|d}t j|d}||_|S)Nrr*) subfolder)r5from_pretrainedrQtuplerr*)clspretrained_model_name_or_pathr8 processorr*r9s r$ruz%InstructBlipProcessor.from_pretrainedse+EGG+,ITTVTT  i ' ' %!! I)9:Wcvwww&7 #r#)N)NNNN)rrr __doc__rmimage_processor_classtokenizer_classqformer_tokenizer_classr6rrrr r rSr rrr_propertyrarl classmethodru __classcell__)r9s@r$r'r'2sC$GFFJL%O-HHHHHH(,^b AA$AI0$y/4HYCZZ[A 45 A AAAAFYYXY&[r#r')rzrgtypingrrimage_processing_utilsr image_utilsrprocessing_utilsrr r tokenization_utils_baser r r utilsrautor get_loggerrloggerrr'__all__r"r#r$rs= """"""""222222%%%%%%HHHHHHHHHHOOOOOOOOOO   H % %"2%"IIIIINIIIX # #r#