`i?!xdZddlZddlmZmZddlmZddlmZm Z m Z m Z m Z ddl mZGdd eZd gZdS) z! Processor class for LayoutLMv2. N)OptionalUnion)ProcessorMixin) BatchEncodingPaddingStrategyPreTokenizedInput TextInputTruncationStrategy) TensorTypec(ReZdZdZddgZdZdZd#fd Z d$d ee e e e e e fd e ee e e fd e ee e e e e e e fde ee e e e e fdedeeeefdeeeefde e de de e de ede edededededede eeefdef&dZdZed Zed!Zed"ZxZS)%LayoutLMv2Processorax Constructs a LayoutLMv2 processor which combines a LayoutLMv2 image processor and a LayoutLMv2 tokenizer into a single processor. [`LayoutLMv2Processor`] offers all the functionalities you need to prepare data for the model. It first uses [`LayoutLMv2ImageProcessor`] to resize document images to a fixed size, and optionally applies OCR to get words and normalized bounding boxes. These are then provided to [`LayoutLMv2Tokenizer`] or [`LayoutLMv2TokenizerFast`], which turns the words and bounding boxes into token-level `input_ids`, `attention_mask`, `token_type_ids`, `bbox`. Optionally, one can provide integer `word_labels`, which are turned into token-level `labels` for token classification tasks (such as FUNSD, CORD). Args: image_processor (`LayoutLMv2ImageProcessor`, *optional*): An instance of [`LayoutLMv2ImageProcessor`]. The image processor is a required input. tokenizer (`LayoutLMv2Tokenizer` or `LayoutLMv2TokenizerFast`, *optional*): An instance of [`LayoutLMv2Tokenizer`] or [`LayoutLMv2TokenizerFast`]. The tokenizer is a required input. image_processor tokenizerLayoutLMv2ImageProcessor)LayoutLMv2TokenizerLayoutLMv2TokenizerFastNc d}d|vr/tjdt|d}||n|}t ||dS)Nfeature_extractorzhThe `feature_extractor` argument is deprecated and will be removed in v5, use `image_processor` instead.)warningswarn FutureWarningpopsuper__init__)selfrrkwargsr __class__s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/layoutlmv2/processing_layoutlmv2.pyrzLayoutLMv2Processor.__init__3st  & ( ( M    !' +> ? ? -<-H//N_ )44444TFrtext text_pairboxes word_labelsadd_special_tokenspadding truncation max_lengthstridepad_to_multiple_ofreturn_token_type_idsreturn_attention_maskreturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mapping return_lengthverbosereturn_tensorsreturnc `|jjr|td|jjr|td|dur|durtd|||}|.|jjr"| t|tr|g}|d}|jdid ||n|dd ||ndd ||n|d d |d |d|d|d| d| d| d| d| d|d|d|d|d|d||}|d}|dur|||d}||d<|S)a This method first forwards the `images` argument to [`~LayoutLMv2ImageProcessor.__call__`]. In case [`LayoutLMv2ImageProcessor`] was initialized with `apply_ocr` set to `True`, it passes the obtained words and bounding boxes along with the additional arguments to [`~LayoutLMv2Tokenizer.__call__`] and returns the output, together with resized `images`. In case [`LayoutLMv2ImageProcessor`] was initialized with `apply_ocr` set to `False`, it passes the words (`text`/``text_pair`) and `boxes` specified by the user along with the additional arguments to [`~LayoutLMv2Tokenizer.__call__`] and returns the output, together with resized `images``. Please refer to the docstring of the above two methods for more information. NzdYou cannot provide bounding boxes if you initialized the image processor with apply_ocr set to True.zaYou cannot provide word labels if you initialized the image processor with apply_ocr set to True.TFzKYou cannot return overflowing tokens without returning the offsets mapping.)imagesr2wordsr!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2 pixel_valuesoverflow_to_sample_mappingimage)r apply_ocr ValueError isinstancestrrrget_overflowing_images)rr5r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2rfeaturesencoded_inputss r__call__zLayoutLMv2Processor.__call__As;D   ) u/@v    ) {/Fs  % , ,1G51P1Pjkk k''vn'UU   4 > 9CT$$$ v )I'   )x/@ #,#8iid !,%%(72C $  21  G  "z "z 6  21 #8"7 #8"7 '@&? (B'A $:#9 (-! "G# $*>'  .n-- $ , ,00Hd9effF"(wr cg}|D]}|||t|t|kr/tdt|dt||S)Nz`Expected length of images to be the same as the length of `overflow_to_sample_mapping`, but got z and )appendlenr<)rr5r8images_with_overflow sample_idxs rr?z*LayoutLMv2Processor.get_overflowing_imagess!4 < rr r rrBr?propertyrNrQr __classcell__)rs@rrrs&$[1J6HO 5 5 5 5 5 5"_cQUIMCG#'5:;@$(,00404*/+0',#;?)TTI0$y/4HYCZZ[TE"3T:K5L"LMN T d49otDcO/DDEF T eDItDI$>?@ T!TtS/12T$%778TSMTT%SMT (~T (~T$(T %)!T"!%#T$%T&'T(!sJ!78)T, -TTTTl $ $ $RRXR**X*$$X$$$$$r r)rVrtypingrrprocessing_utilsrtokenization_utils_baserrr r r utilsr r__all__r:r rrcs""""""""......wwwwwwwwwwwwww\$\$\$\$\$.\$\$\$~ ! !r