`i) ddlmZmZmZmZddlmZmZmZm Z m Z ddl m Z m Z erddlmZddlmZmZerddlZdd lmZdd lmZe jeZee d Gd de ZdS))AnyOptionalUnionoverload)add_end_docstringsis_torch_availableis_vision_availableloggingrequires_backends) ChunkPipelinebuild_pipeline_init_args)Image) load_image valid_imagesN)BaseModelOutput)2MODEL_FOR_ZERO_SHOT_OBJECT_DETECTION_MAPPING_NAMEST)has_image_processorceZdZdZdZdZdZdZfdZe de e dfde e e e fde d e ee e ffd Ze de ee e fde d e e ee e ffd Z dde e de ee e ffdee e e e fde d e e ee e fe e ee e ffffd ZdZddZdZddZddd ee effdZxZS)ZeroShotObjectDetectionPipelinea Zero shot object detection pipeline using `OwlViTForObjectDetection`. This pipeline predicts bounding boxes of objects when you provide an image and a set of `candidate_labels`. Example: ```python >>> from transformers import pipeline >>> detector = pipeline(model="google/owlvit-base-patch32", task="zero-shot-object-detection") >>> detector( ... "http://images.cocodataset.org/val2017/000000039769.jpg", ... candidate_labels=["cat", "couch"], ... ) [{'score': 0.287, 'label': 'cat', 'box': {'xmin': 324, 'ymin': 20, 'xmax': 640, 'ymax': 373}}, {'score': 0.254, 'label': 'cat', 'box': {'xmin': 1, 'ymin': 55, 'xmax': 315, 'ymax': 472}}, {'score': 0.121, 'label': 'couch', 'box': {'xmin': 4, 'ymin': 0, 'xmax': 642, 'ymax': 476}}] >>> detector( ... "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", ... candidate_labels=["head", "bird"], ... ) [{'score': 0.119, 'label': 'bird', 'box': {'xmin': 71, 'ymin': 170, 'xmax': 410, 'ymax': 508}}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This object detection pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"zero-shot-object-detection"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=zero-shot-object-detection). FTc tjdi||jdkrtd|jdt |d|tdS)NtfzThe z is only available in PyTorch.vision)super__init__ framework ValueError __class__r check_model_typer)selfkwargsr s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/pipelines/zero_shot_object_detection.pyrz(ZeroShotObjectDetectionPipeline.__init__=st""6""" >T ! !RDNRRRSS S$))) PQQQQQimagez Image.Imagecandidate_labelsr#returnc dSNr)r"r&r'r#s r$__call__z(ZeroShotObjectDetectionPipeline.__call__Fs  #sr%c dSr*r)r"r&r#s r$r+z(ZeroShotObjectDetectionPipeline.__call__Ksbeber%Nc d|vr|d}t|ttjfr||d}nmt|tt frNt |r?t tjdt||Dfi|S |}tj|fi|}|S)a| Detect objects (bounding boxes & classes) in the image(s) passed as inputs. Args: image (`str`, `PIL.Image` or `list[dict[str, Any]]`): The pipeline handles three types of images: - A string containing an http url pointing to an image - A string containing a local path to an image - An image loaded in PIL directly You can use this parameter to send directly a list of images, or a dataset or a generator like so: ```python >>> from transformers import pipeline >>> detector = pipeline(model="google/owlvit-base-patch32", task="zero-shot-object-detection") >>> detector( ... [ ... { ... "image": "http://images.cocodataset.org/val2017/000000039769.jpg", ... "candidate_labels": ["cat", "couch"], ... }, ... { ... "image": "http://images.cocodataset.org/val2017/000000039769.jpg", ... "candidate_labels": ["cat", "couch"], ... }, ... ] ... ) [[{'score': 0.287, 'label': 'cat', 'box': {'xmin': 324, 'ymin': 20, 'xmax': 640, 'ymax': 373}}, {'score': 0.25, 'label': 'cat', 'box': {'xmin': 1, 'ymin': 55, 'xmax': 315, 'ymax': 472}}, {'score': 0.121, 'label': 'couch', 'box': {'xmin': 4, 'ymin': 0, 'xmax': 642, 'ymax': 476}}], [{'score': 0.287, 'label': 'cat', 'box': {'xmin': 324, 'ymin': 20, 'xmax': 640, 'ymax': 373}}, {'score': 0.254, 'label': 'cat', 'box': {'xmin': 1, 'ymin': 55, 'xmax': 315, 'ymax': 472}}, {'score': 0.121, 'label': 'couch', 'box': {'xmin': 4, 'ymin': 0, 'xmax': 642, 'ymax': 476}}]] ``` candidate_labels (`str` or `list[str]` or `list[list[str]]`): What the model should recognize in the image. threshold (`float`, *optional*, defaults to 0.1): The probability necessary to make a prediction. top_k (`int`, *optional*, defaults to None): The number of top predictions that will be returned by the pipeline. If the provided number is `None` or higher than the number of predictions available, it will default to the number of predictions. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A list of lists containing prediction results, one list per input image. Each list contains dictionaries with the following keys: - **label** (`str`) -- Text query corresponding to the found object. - **score** (`float`) -- Score corresponding to the object (between 0 and 1). - **box** (`dict[str,int]`) -- Bounding box of the detected object in image's original size. It is a dictionary with `x_min`, `x_max`, `y_min`, `y_max` keys. text_queriesr&r'c3&K|] \}}||dV dS)r/Nr).0imglabelss r$ z;ZeroShotObjectDetectionPipeline.__call__..s.ppKCs??ppppppr%) pop isinstancestrrlisttuplerrr+zip)r"r&r'r#inputsresultsr s r$r+z(ZeroShotObjectDetectionPipeline.__call__Ns~ V # #%zz.99  ec5;/ 0 0 $:JKKFF e} - - ,u2E2E   ppSVW\^nSoSoppp  F"%''"644V44r%c ni}d|vr |d|d<i}d|vr |d|d<d|vr |d|d<|i|fS)Ntimeout thresholdtop_kr)r"r#preprocess_paramspostprocess_paramss r$_sanitize_parametersz4ZeroShotObjectDetectionPipeline._sanitize_parameterssl   +1)+< i ( & .4[.A { + f  *0/ w ' "&888r%c#Kt|d|}|d}t|tr|d}t j|j|jggtj}t|D]\}}| ||j }| ||j } |j dkr| |j} |t|dz k||d || VdS) Nr&)r>r',)dtype)return_tensorsptr )is_last target_sizecandidate_label)rr6r7splittorchtensorheightwidthint32 enumerate tokenizerrimage_processortorFlen) r"r;r>r&r'rJirK text_inputsimage_featuress r$ preprocessz*ZeroShotObjectDetectionPipeline.preprocesss96'?G<<<!"45 & , , ;/55c:: lU\5;$?#@ TTT "+,<"="=   A...XXK!11%1WWN~%%!/!2!24:!>!>$4 5 5 99*#2  !       r%c|d}|d}|d}|jdi|}|||d|}|S)NrJrKrI)rJrKrIr)r5model)r" model_inputsrJrKrIoutputs model_outputss r$_forwardz(ZeroShotObjectDetectionPipeline._forwardso"&&}55 &**+<==""9--$*,,|,,(3dkwwovw r%皙?cg}|D]}|d}t|}|j|||dd}|dD]d}|d|} ||d|d} | || d} || et|dd }|r |d|}|S) NrKrJ)r^r? target_sizesrscoresboxes)scorelabelboxc|dS)Nrfr)xs r$z=ZeroShotObjectDetectionPipeline.postprocess..s ' r%T)keyreverse)rrTpost_process_object_detectionnonzeroitem_get_bounding_boxappendsorted) r"r_r?r@r< model_outputrgr^indexrfrhresults r$ postprocessz+ZeroShotObjectDetectionPipeline.postprocesss) ' 'L !23E*<88L*HH$ UbHcIG!*2244 ' ')%05577,,WW-=e-DQ-GHH#(5EEv&&&&  '&:&:DIII  &fufoGr%rhz torch.Tensorc|jdkrtd|\}}}}||||d}|S)a% Turns list [xmin, xmax, ymin, ymax] into dict { "xmin": xmin, ... } Args: box (`torch.Tensor`): Tensor containing the coordinates in corners format. Returns: bbox (`dict[str, int]`): Dict containing the coordinates in corners format. rHzAThe ZeroShotObjectDetectionPipeline is only available in PyTorch.)xminyminxmaxymax)rrinttolist)r"rhryrzr{r|bboxs r$rqz1ZeroShotObjectDetectionPipeline._get_bounding_boxsa >T ! !`aa a!$!1!1!3!3dD$     r%r*)raN)__name__ __module__ __qualname____doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerrrrr7r8rdictr+rrCrZr`rwr}rq __classcell__)r s@r$rrs/@O #ORRRRR#3 -.#BGTRUYBW#cf# d38n ###X#ed4S>2eced4PTUXZ]U]P^K_F`eeeXe =AVVS-d38n)==>V#5d3i#89V V tDcN#T$tCH~*>%?? @ VVVVVVp 9 9 9(,^S#Xr%r)typingrrrrutilsrr r r r baserrPILr image_utilsrrrMtransformers.modeling_outputsrmodels.auto.modeling_autor get_loggerrloggerrrr%r$rs_111111111111kkkkkkkkkkkkkk99999999766666666_LLL======^^^^^^  H % %,,FFFGGaaaaamaaHGaaar%