Pi[ddlmZmZmZmZmZmZddlZddlm cm Z ddl m Z ddlmZmZddlmZ ddeejded ed ejfd Zd eeeeefd edeedeeeeefffdZdefd eeeeefdeded eeejffdZdedddfd eeeefdeded edeedeed eeejffdZd eed eeejffdZdefd eeeeefdeded eejejffdZdS))AnyDictListOptionalTupleUnionN pad_sequence)CROSS_ENTROPY_IGNORE_IDX PACK_TYPE)packed_block_causal_maskF sequences batch_first padding_valuereturncttd|||t|gS)a This function is identical to :func:`torch.nn.utils.rnn.pad_sequence`, but instead pads a list of variable length Tensors from the left to the length of the longest sequence. Note: This function returns a Tensor of size ``T x B x *`` or ``B x T x *`` where `T` is the length of the longest sequence. This function assumes trailing dimensions and type of all the Tensors in sequences are same. Args: sequences (List[torch.Tensor]): list of variable length sequences. batch_first (bool): if ``True``, the output will be in ``B x T x *`` format, ``T x B x *`` otherwise. Default False. padding_value (float): value for padded elements. Default: 0. Returns: Tensor of size ``T x B x *`` if :attr:`batch_first` is ``False``. Tensor of size ``B x T x *`` otherwise Example: >>> a = torch.tensor([1, 2, 3]) >>> b = torch.tensor([4, 5, 6, 7]) >>> c = torch.tensor([8, 9, 10, 11, 12]) >>> left_pad_sequence([a, b, c], batch_first=True, padding_value=0) tensor([[ 0, 0, 1, 2, 3], [ 0, 4, 5, 6, 7], [ 8, 9, 10, 11, 12]]) c0tj|dgS)Nrdims)torchflip)xs k/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchtune/data/_collate.pyz#left_pad_sequence..2sej!---rrr)r maprint)rrrs rleft_pad_sequencersOD  - -y99#    d[!!"d## $rbatch pad_direction keys_to_pad padding_idxc p|dvrtd|ttrstddtt|trt|ks(td|ddtdks;td d tdd fd dD}fd |D}|dkrt jjj j nt}D]<|fdDdt|tr|n||<=|S)a A generic padding collation function which pads ``keys_to_pad`` entries in a batch of sequences from the given ``pad_direction`` to the maximum sequence length for each entry in the batch. Note: This function assumes all batch elements which are not in ``keys_to_pad`` do not require any collation (see example below). Args: batch (List[Dict[str, List[int]]]): A list of dictionaries containing inputs. pad_direction (str): whether to pad entries from the left, or right. If ``pad_direction="right"``, we use :func:`torch.nn.utils.rnn.pad_sequence`, otherwise if ``pad_direction="left"``, we use :func:`torchtune.data.left_pad_sequence`. keys_to_pad (List[str]): Batch element keys to apply padding to. Should be a subset of keys in the batch. padding_idx (Union[int, Dict[str, int]]): Either a single integer padding value to apply to all ``keys_to_pad`` elements, or a mapping with keys identical to ``keys_to_pad`` with per-key padding values. Returns: torch.Tensor: The padded tensor of input ids with shape ``[batch_size, max_seq_len]``. Raises: ValueError: If ``pad_direction`` is not one of "left" or "right", **or** if ``keys_to_pad`` is empty, or is not a list, **or** if ``keys_to_pad`` is not a subset of keys in the batch, **or** if ``padding_idx`` is provided as a dictionary, but the keys are not identical to ``keys_to_pad`` Example: >>> a = [1, 2, 3] >>> b = [4, 5, 6, 7] >>> c = [8, 9, 10, 11, 12] >>> batch = [ >>> {"tokens": a, "labels": 1}, >>> {"tokens": b, "labels": 3}, >>> {"tokens": c, "labels": 0}, >>> ] >>> padded_collate( >>> batch, >>> pad_direction="left", >>> keys_to_pad=["tokens"], >>> padding_idx=-10 >>> ) { 'labels': tensor([1, 3, 0]), 'tokens': tensor([[-10, -10, 1, 2, 3], [-10, 4, 5, 6, 7], [ 8, 9, 10, 11, 12]]) } leftright;pad_direction should be one of 'left' or 'right' but found zMkeys_to_pad should be a list of strings with at least one element, but found !z8padding_idx was provided as a dictionary, but the keys (z#) are not the same as keys_to_pad ()rz?keys_to_pad should be a subset of keys in the batch, but found z and z, respectively.cg|]}|v| Sr,).0kr"s r z"padded_collate..s#EEE0D0D!0D0D0DrcTi|]#tjfdD$S)c g|] }| Sr,r,r-rr.s rr/z-padded_collate...s#8#8#8QAaD#8#8#8rrtensor)r-r.r s @r z"padded_collate..s;NNNa1el#8#8#8#8%#8#8#899NNNrr'cDg|]}tj|Sr,r3r2s rr/z"padded_collate..s' / / /AU\!A$   / / /rTr) ValueError isinstancelistsetdictkeysrnnutilsrnnr r)r r!r"r# batch_keys output_dictpad_fnr.s` ` @rpadded_collaterC8s6v--- Y- Y Y    k4 ( (   j\g j j j   k""K+t$$ ;##%%&&+55C;K[K[K]K]CC4?CCC c%(--//2222KKK%(q%9%9KKK FEEEU1X]]__EEEJNNNN:NNNK G # # ''     / / / / / / /",[$"?"?P A[    A r ignore_idxctd|Dd|}td|Dd|}|jd}|jd}||krtj|d||z f|}n"||krtj|d||z f|}||dS) aPad a batch of sequences to the longest sequence length in the batch, and convert integer lists to tensors. Args: batch (List[Dict[str, List[int]]]): A list of dictionaries containing input, label pairs. padding_idx (int): Padding index for input ids. Defaults to 0. ignore_idx (int): Padding index for labels. Defaults to -100. Returns: Dict[str, torch.Tensor]: Collated input and label tensors. Example: >>> token_pairs = [ >>> {"tokens": [1, 2, 3], "labels": [4, 5, 6]}, >>> {"tokens": [7,], "labels": [10,]}, >>> ] >>> collated = padded_collate( >>> batch=token_pairs, >>> padding_idx=padding_idx, >>> ignore_idx=ignore_idx, >>> ) >>> collated["tokens"] >>> tensor([[1, 2, 3], [7, 0, 0]]) >>> collated["labels"] >>> tensor([[4, 5, 6], [10, -100, -100]]) cBg|]}tj|dStokensr3r-rs rr/z&padded_collate_sft..&222qak " "222rTrcBg|]}tj|dSlabelsr3rIs rr/z&padded_collate_sft..rJrrvaluerHrM)r shapeFpadlong)r r#rD input_idsrMinput_ids_seq_lenlabels_seq_lens rpadded_collate_sftrYs>22E222!I 22E222 F "+\"%N>)) Q)N:;:    + + +E  !22 3    nn&&&++-- @ @@rr' pad_max_tilespad_max_imagesc ^|dvrtd||dkrd|D}t|||}n$|dkrdtd|Dd| i}|djd }t |} t d |D} ||| krtd | d||} g} g} g} |D]R}g}g}t |dd|dD]\}}|jd}|j\}}||z}| |z }|dkr||z nd}|dkr||z nd}tj|ddddddd|fd}tj|d||z||fd}| || || tj || tj || tj |ddTt| dd }t| dd }t| dd }| | |d }|+|j\} } }!tj|d|| z|z|!z f}|d||d|d}"d|vr |d|"d<|"S)a$Pad a batch of text sequences, tiled image tensors, aspect ratios, and cross attention masks. This can be used for both training and inference. ``batch`` is expected to be a list of sample dicts containing the following:: - "tokens": List[int] of length text_seq_len, varies across samples - "labels": List[int] of length text_seq_len, varies across samples - "encoder_input": Dict[str, List[torch.Tensor]] - "images": List[torch.Tensor], each with shape (n_tiles, c, h, w) - "aspect_ratio": List[torch.Tensor], each with shape (2, ) to indicate h_ratio, w_ratio - "encoder_mask": List[Tensor], each with shape (text_seq_len, image_seq_len) Shape notation: - c = channel dim - h = height dim - w = weight dim Note: For each element in the batch, ``len(images) == len(encoder_mask) == len(aspect_ratio)``. This collater does the following: (1) Pad text sequence and encoder mask to the longest sequence length in the batch (2) Pad image tensors in the tile dimension with zeros to the largest number of tiles in the batch (3) Add empty images of zeros to samples up to max number of images in the batch (4) Pad aspect ratios with (1,1) for all added padding images Args: batch (List[Dict[str, Any]]): A list of sample dicts containing tokens, labels, images, encoder_mask, and aspect_ratio. padding_idx (int): Padding index for input token ids. Defaults to 0. ignore_idx (int): Padding index for labels. Defaults to -100. pad_direction (str): whether to pad entries from the left, or right. If ``pad_direction="right"``, we use :func:`torch.nn.utils.rnn.pad_sequence`, otherwise if ``pad_direction="left"``, we use :func:`torchtune.data.left_pad_sequence`. For training, we typically want to pad from the right. For inference, we typically want to pad from the left. Defaults to "right". pad_max_tiles (Optional[int]): Maximum number of tiles to pad to. If None, will pad to the largest number of tiles in the batch. Defaults to None. pad_max_images (Optional[int]): Maximum number of images to pad to. If None, will pad to the largest number of images in the batch. Defaults to None. Returns: Dict[str, Tensor]: Collated tokens, labels, images, encoder_mask, aspect_ratio tensors. - tokens: Tensor of shape (bsz, max_seq_len) - labels: Tensor of shape (bsz, max_seq_len) - images: Tensor of shape (bsz, max_num_images, max_num_tiles, c, h, w) - encoder_mask: Tensor of shape (bsz, max_seq_len, tokens_per_tile * max_num_tiles * max_num_images) - aspect_ratio: Tensor of shape (bsz, max_num_images, 2) Raises: ValueError: If ``pad_direction`` is not one of "left" or "right", **or** if pad_max_tiles is set to a value less than the largest number of tiles in an image. Example: >>> image_id = 1 >>> tokens_per_tile = 5 >>> c, h, w = 1, 1, 1 >>> batch = [ ... { ... "tokens": [1, 2, 1, 3], "labels": [4, 5, 6, 7], ... "encoder_input": { ... # One image with two tiles, one image with three tiles ... "images": [torch.ones(2, c, h, w), torch.ones(3, c, h, w)], ... "aspect_ratio": [torch.tensor([1, 2]), torch.tensor([1, 3])], ... }, ... # Mask is shape (text_seq_len, tokens_per_tile * n_tiles) ... "encoder_mask": [torch.ones(4, 5 * 2), torch.ones(4, 5 * 3)], ... }, ... { ... "tokens": [1, 4], "labels": [8, 9], ... "encoder_input": { ... # One image with four tiles ... "images": [torch.ones(4, c, h, w)], ... "aspect_ratio": [torch.tensor([2, 2])], ... }, ... # Mask is shape (text_seq_len, tokens_per_tile * n_tiles) ... "encoder_mask": [torch.ones(2, 5 * 4)], ... }, ... ] >>> model_inputs = padded_collate_tiled_images_and_mask(batch=batch) >>> print(model_inputs["tokens"]) tensor([[1, 2, 1, 3], [1, 4, 0, 0]]) >>> print(model_inputs["labels"]) tensor([[4, 5, 6, 7], [8, 9, -100, -100]]) >>> print(model_inputs["encoder_input"]["images"].shape) # (bsz, max_num_images, max_num_tiles, c, h, w) torch.Size([2, 2, 4, 1, 1, 1]) >>> print(model_inputs["encoder_mask"].shape) # (bsz, max_text_seq_len, tokens_per_tile * max_num_tiles * max_num_images) torch.Size([2, 4, 40]) >>> print(model_inputs["encoder_input"]["aspect_ratio"].shape) # (bsz, max_num_images, 2) torch.Size([2, 2, 2]) >>> print(model_inputs["encoder_input"]["images"][0, 0, ...]) # Image with two tiles got padded to four tensor([[[[1.]]], [[[1.]]], [[[0.]]], [[[0.]]]]) >>> print(model_inputs["encoder_input"]["images"][0, 1, ...]) # Image with three tiles got padded to four tensor([[[[1.]]], [[[1.]]], [[[1.]]], [[[0.]]]]) >>> print(model_inputs["encoder_input"]["images"][1, 0, ...]) # Image with four tiles did not get padded tensor([[[[1.]]], [[[1.]]], [[[1.]]], [[[1.]]]]) >>> print(model_inputs["encoder_input"]["images"][1, 1, ...]) # Extra padding image was added to second sample tensor([[[[0.]]], [[[0.]]], [[[0.]]], [[[0.]]]]) r%r(r'c0g|]}|d|ddS)rHrMrQr,)r-samples rr/z8padded_collate_tiled_images_and_mask..Qs7   IOvh'6(3C D D   rr&rHcBg|]}tj|dSrGr3rIs rr/z8padded_collate_tiled_images_and_mask..Ys&:::qak**:::rTrrNc3RK|]"}|ddD]}|jdV#dS) encoder_inputimagesrN)rR)r-r^images r z7padded_collate_tiled_images_and_mask..gs] O,X6   ArNzMore tiles in image z, than pad_max_tiles rarb encoder_maskrrO aspect_ratio)rbrf)rHrarerM)r7rYrrRlenmaxziprSrTappendrstackr view)#r r#rDr!rZr[ text_only collated_text max_seq_lenbsz max_num_tiles batch_images batch_masksbatch_aspect_ratiosr^ sample_images sample_masksrcmaskn_tiles text_seq_len image_seq_lentokens_per_tile padding_tilesright_padding_textleft_padding_text padded_image padded_maskcollated_imagescollated_maskscollated_aspect_ratios concat_masks_img_seq batch_dicts# r$padded_collate_tiled_images_and_maskrsZ--- Y- Y Y     SX   +9k:NN & '::E::: )  )/3K e**CM  = ( (Z}ZZ=ZZ & LK(Y(Y   ? #H -vn/E  ! -! -KE4k!nG+/* 'L-+w6O)G3M.;w.F.F l**A />> token_pairs = [ >>> {"tokens": [1, 2, 3, 4, 5, 6], "labels": [7, 8, 9, 10, 11, 12], >>> "input_pos": [0, 1, 2, 0, 1, 0], "seq_lens": [3, 2, 1]}, >>> {"tokens": [13, 14, 15, 16, 17, 18], "labels": [19, 20, 21, 22, 23, 24], >>> "input_pos": [0, 1, 0, 1, 0, 1], "seq_lens": [2, 2, 2]}, >>> ] >>> collated = padded_collate_packed( >>> batch=token_pairs, >>> device=device, >>> ) >>> collated["mask"] >>> tensor([ >>> [[1, 0, 0, 0, 0, 0], >>> [1, 1, 0, 0, 0, 0], >>> [1, 1, 1, 0, 0, 0], >>> [0, 0, 0, 1, 0, 0], >>> [0, 0, 0, 1, 1, 0], >>> [0, 0, 0, 0, 0, 1]], >>> [[1, 0, 0, 0, 0, 0], >>> [1, 1, 0, 0, 0, 0], >>> [0, 0, 1, 0, 0, 0], >>> [0, 0, 1, 1, 0, 0], >>> [0, 0, 0, 0, 1, 0], >>> [0, 0, 0, 0, 1, 1]]) cg|] }|d SrGr,rIs rr/z)padded_collate_packed..555!!H+555rcg|] }|d SrLr,rIs rr/z)padded_collate_packed..rrcg|] }|d S) input_posr,rIs rr/z)padded_collate_packed..s;;;Q{^;;;rcg|] }|d Sseq_lensr,rIs rr/z)padded_collate_packed..s---!* ---rr)rHrMrrx)rrlr )r rHrMrr block_masks rpadded_collate_packedrsX[55u555 6 6F [55u555 6 6F ;;U;;;<>> batch = [ >>> {'chosen_input_ids': [1, 2, 3], 'rejected_input_ids': [4, 5], >>> 'chosen_labels': [6, 7, 8], 'rejected_labels': [9, 10]}, >>> {'chosen_input_ids': [11, 12], 'rejected_input_ids': [13, 14, 15], >>> 'chosen_labels': [16, 17], 'rejected_labels': [18, 19, 20]}, >>> ] >>> padded_collate_dpo(batch) >>> (tensor([[ 1, 2, 3], >>> [11, 12, 0], >>> [ 4, 5, 0], >>> [13, 14, 15]]), >>> tensor([[ 6, 7, 8], >>> [16, 17, -100], >>> [ 9, 10, -100], >>> [18, 19, 20]])) cBg|]}tj|dS)chosen_input_idsr3r-exs rr/z&padded_collate_dpo..&s(MMM R(:%;<<MMMrcBg|]}tj|dS)rejected_input_idsr3rs rr/z&padded_collate_dpo..'s(QQQR%,r*>'?@@QQQrcBg|]}tj|dS) chosen_labelsr3rs rr/z&padded_collate_dpo..(s'GGG2U\"_"566GGGrcBg|]}tj|dS)rejected_labelsr3rs rr/z&padded_collate_dpo..)s(KKKru|B'8$9::KKKrTrr ) r r#rDrrrrto_pad_input_ids to_pad_labelsconcatenated_input_idsconcatenated_labelss rpadded_collate_dporsNNMuMMMQQ5QQQGGGGGMKKUKKKO'*<<!O3M)d+'4z "#6 66r)Fr)typingrrrrrrrtorch.nn.functionalr= functionalrStorch.nn.utils.rnnr torchtune.data._commonr r !torchtune.modules.attention_utilsr TensorboolfloatrstrrrCrYrrrr,rrrs ;::::::::::::::: ++++++FFFFFFFFFFFFFF &$&$EL!&$&$&$ \ &$&$&$&$Re S$s)^$ %eec e sDcN*+ eeeeT.8A8A S$s)^$ %8A8A8A #u|  8A8A8A8A~. #'$( bb S#X bbb b C= b SM b #u| bbbbJ:  ?: #u| ::::~.6767 S$s)^$ %676767 5< %& 676767676767r