`i-dZddlmZddlmZmZddlZddlmZddlm Z ddl m Z dd l m Z dd lmZmZmZmZdd lmZdd lmZd dlmZejeZeedGddeZeGdde ZddZGddejZ GddejZ!edGddeZ"ddgZ#dS)zPyTorch VitPose model.) dataclass)OptionalUnionN)nn)BackboneOutput)PreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstringlogging) load_backbone)can_return_tuple) VitPoseConfigz6 Class for outputs of pose estimation models. ) custom_introceZdZUdZdZeejed<dZ eejed<dZ ee ejdfed<dZ ee ejdfed<dS)VitPoseEstimatorOutputaH loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Loss is not supported at this moment. See https://github.com/ViTAE-Transformer/ViTPose/tree/main/mmpose/models/losses for further detail. heatmaps (`torch.FloatTensor` of shape `(batch_size, num_keypoints, height, width)`): Heatmaps as predicted by the model. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states (also called feature maps) of the model at the output of each stage. Nlossheatmaps. hidden_states attentions) __name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrtupler/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/vitpose/modeling_vitpose.pyrr%s  )-D(5$ %,,,,0Hhu()000=AM8E%"3S"89:AAA:>Ju0#567>>>>>r#rcbeZdZUeed<dZdZdZdee j e j e j ffdZ dS)VitPosePreTrainedModelconfigvit pixel_valuesTmodulecJt|tjtjfrtj|jjtj d|j j |jj |j_|j |jjdSdSt|tjr?|jj|jjddSdS)zInitialize the weightsg)meanstdNg?) isinstancerLinearConv2dinit trunc_normal_weightdatatorfloat32r'initializer_rangedtypebiaszero_ LayerNormfill_)selfr*s r$ _init_weightsz$VitPosePreTrainedModel._init_weightsDs fry")4 5 5 *"$!6!6 "%%em443DKDa"7""b$%% M {&  &&((((('&  - - * K  " " $ $ $ M  $ $S ) ) ) ) ) * *r#N)rrrrr base_model_prefixmain_input_namesupports_gradient_checkpointingrrr/r0r;r>r"r#r$r&r&=s` $O&*# *E")RY *L$M * * * * * *r#r&gaussian-heatmapc|dvrtd|jdkrtd|j\}}}}d}|dkrd}|dddddd f |dddddd f<||d |||}|}|D]/\} } |dd| d f|dd| d f<|dd| d f|dd| d f<0|||||f}|d }|S) aFlip the flipped heatmaps back to the original form. Args: output_flipped (`torch.tensor` of shape `(batch_size, num_keypoints, height, width)`): The output heatmaps obtained from the flipped images. flip_pairs (`torch.Tensor` of shape `(num_keypoints, 2)`): Pairs of keypoints which are mirrored (for example, left ear -- right ear). target_type (`str`, *optional*, defaults to `"gaussian-heatmap"`): Target type to use. Can be gaussian-heatmap or combined-target. gaussian-heatmap: Classification target with gaussian distribution. combined-target: The combination of classification target (response map) and regression target (offset map). Paper ref: Huang et al. The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation (CVPR 2020). Returns: torch.Tensor: heatmaps that flipped back to the original image )rBcombined-targetz9target_type should be gaussian-heatmap or combined-targetzCoutput_flipped should be [batch_size, num_keypoints, height, width]rrDrN.) ValueErrorndimshapereshapeclonetolistflip) output_flipped flip_pairs target_type batch_size num_keypointsheightwidthchannelsoutput_flipped_backleftrights r$ flip_backrYSsi"AAATUUUa^___/=/C,J vuH'''(6qqq!$Q$|(D'Dqqq!$Q$|$#++JHfeTTN(..00"((**JJ e,:111eS=,IAAAtSL)-;AAAtSL-IAAAucM**-55z=RXZ_6`aa-22266 r#cheZdZdZdeffd Zd dejdeejdejfdZ xZ S) VitPoseSimpleDecoderz Simple decoding head consisting of a ReLU activation, 4x upsampling and a 3x3 convolution, turning the feature maps into heatmaps. r'cttj|_tj|jdd|_tj|j j |j ddd|_ dS)NbilinearF) scale_factormode align_cornersrr kernel_sizestridepadding) super__init__rReLU activationUpsampler^ upsamplingr0backbone_config hidden_size num_labelsconvr=r' __class__s r$rfzVitPoseSimpleDecoder.__init__st '))+63FZglmmmI  " .0AqYZde    r#N hidden_staterOreturnc||}||}||}|t||}|SN)rhrjrnrYr=rqrOrs r$forwardzVitPoseSimpleDecoder.forwardsO|44 |44 99\**  ! :66Hr#rt rrrrrrfrTensorrrv __classcell__rps@r$r[r[{s  }        EL hu|>T `e`l        r#r[cZeZdZdZdeffd ZddejdeejfdZ xZ S) VitPoseClassicDecoderz Classic decoding head consisting of a 2 deconvolutional blocks, followed by a 1x1 convolution layer, turning the feature maps into heatmaps. r'cttj|jjddddd|_tjd|_tj |_ tjdddddd|_ tjd|_ tj |_ tjd|jddd|_dS) NrErF)rbrcrdr9rra)rerfrConvTranspose2drkrldeconv1 BatchNorm2d batchnorm1rgrelu1deconv2 batchnorm2relu2r0rmrnros r$rfzVitPoseClassicDecoder.__init__s )  " .1VW^c    .--WYY )#s!UV]bccc .--WYY Ic6#4!AWXYYY r#NrqrOcP||}||}||}||}||}||}||}|t||}|Srt)rrrrrrrnrYrus r$rvzVitPoseClassicDecoder.forwards||L11 |44 zz,// ||L11 |44 zz,// 99\**  ! :66Hr#rtrwrzs@r$r|r|s Z} Z Z Z Z Z ZELhu|>Tr#r|z? The VitPose model with a pose estimation head on top. ceZdZdeffd Zee d dejde ejde ejde ejde e d e f d Z xZS) VitPoseForPoseEstimationr'ct|t||_t |jjdst dt |jjdst dt |jjdst d|jrt|nt||_ | dS)Nrlz0The backbone should have a hidden_size attribute image_sizez0The backbone should have an image_size attribute patch_sizez/The backbone should have a patch_size attribute) rerfrbackbonehasattrr'rGuse_simple_decoderr[r|head post_initros r$rfz!VitPoseForPoseEstimation.__init__s    %f-- t}+];; QOPP Pt}+\:: QOPP Pt}+\:: PNOO O4:4Mp(000ShioSpSp  r#Nr) dataset_indexrOlabelskwargsrrc ,d}|td|jj|fd|i|}|jd}|jd} |jjjd|jjjdz} |jjjd|jjjdz} | ddd}| | d| |  }| ||} t|| |j|j S) ac dataset_index (`torch.Tensor` of shape `(batch_size,)`): Index to use in the Mixture-of-Experts (MoE) blocks of the backbone. This corresponds to the dataset index used during training, e.g. For the single dataset index 0 refers to the corresponding dataset. For the multiple datasets index 0 refers to dataset A (e.g. MPII) and index 1 refers to dataset B (e.g. CrowdPose). flip_pairs (`torch.tensor`, *optional*): Whether to mirror pairs of keypoints (for example, left ear -- right ear). Examples: ```python >>> from transformers import AutoImageProcessor, VitPoseForPoseEstimation >>> import torch >>> from PIL import Image >>> import requests >>> processor = AutoImageProcessor.from_pretrained("usyd-community/vitpose-base-simple") >>> model = VitPoseForPoseEstimation.from_pretrained("usyd-community/vitpose-base-simple") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> boxes = [[[412.8, 157.61, 53.05, 138.01], [384.43, 172.21, 15.12, 35.74]]] >>> inputs = processor(image, boxes=boxes, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> heatmaps = outputs.heatmaps ```NzTraining is not yet supportedrrFrrr)rO)rrrr)NotImplementedErrorrforward_with_filtered_kwargs feature_mapsrIr'rkrrpermuterJ contiguousrrrr) r=r)rrOrrroutputssequence_outputrQ patch_height patch_widthrs r$rvz VitPoseForPoseEstimation.forwards0N  %&EFF F"L$-"L # # '# # # ".r2$*1- {2=a@DKD_DjklDmm k1 > l>   -> U\* > & > +, >  > > > ^> > > > > r#r)rB)$r dataclassesrtypingrrrrmodeling_outputsrmodeling_utilsr processing_utilsr utilsr r r rutils.backbone_utilsr utils.genericrconfiguration_vitposer get_loggerrloggerrr&rYModuler[r|r__all__r"r#r$rs]!!!!!!"""""""" ......------&&&&&&MMMMMMMMMMMM111111------000000  H % %   ?????[??   ?$*****_****%%%%P296#####BI###L S S S S S 5S S  S l $%? @r#