PiddlmZmZmZmZmZmZmZmZddl m Z ddl m Z m Z ddlmZddlmZedd ZGd d e e Zd eee fd e ede e fdZGdde e ZdS))AnyCallableDictIterableIteratorMappingOptionalTypeVar)Sampler)BaseNodeT)Mapper)StatefulKT) covariantceZdZdZdZdZdeeffd Zd de e e e fffd Z defd Zde e e ffd ZxZS) IterableWrappera Thin Wrapper that converts any Iterable (including torch.utils.data.IterableDataset) in to a BaseNode. If iterable implements the Stateful Protocol, it will be saved and restored with its state_dict/load_state_dict methods. Args: iterable (Iterable[T]): Iterable to convert to BaseNode. IterableWrapper calls iter() on it. :warning: Note the distinction between state_dict/load_state_dict defined on Iterable, vs Iterator. Only the Iterable's state_dict/load_state_dict are used. _num_yieldediterablecdt||_d|_dSN)super__init__r_it)selfr __class__s l/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchdata/nodes/adapters.pyrzIterableWrapper.__init__&s+   *.N initial_statec Td|_d|_t||||j|_t |jtr@|j||j t|j|_dSt|j|_t|jD]@} t|j#t$rtd|jd|dwxYwdSt|j|_dS)NrTried to fast-forward / items during init but hit StopIteration after 4 items, this is likely a bug or malformed state_dict)rrrresetNUM_YIELDED_KEY isinstancerrload_state_dict ITERABLE_KEYiterrangenext StopIteration ValueErrorrr irs rr%zIterableWrapper.reset+s>  m$$$  $ -d.B CD $-22  --mDrr map_datasetsamplerr1cNt|}t||j}|S)aThin Wrapper that converts any MapDataset in to a torchdata.node If you want parallelism, copy this and replace Mapper with ParallelMapper. Args: map_dataset (Mapping[K, T]): - Apply map_dataset.__getitem__ to the outputs of sampler. sampler (Sampler[K]): )SamplerWrapperr __getitem__)rArB sampler_node mapper_nodes rMapStyleWrapperrHNs)'5W&=&=L{'>??K rc eZdZdZdZdZdZ ddeede d e e e ge fffd Z dd e e eefffd Zd efdZd e eeffdZede d e fdZxZS)rDa Convert a sampler into a BaseNode. This is nearly identical to IterableWrapper except it includes a hook to call set_epoch on the sampler, if it supports it. Args: sampler (Sampler): Sampler to wrap. initial_epoch (int): initial epoch to set on the sampler epoch_updater (Optional[Callable[[int], int]] = None): callback to update epoch at start of new iteration. It's called at the beginning of each iterator request, except the first one. r_epoch_samplerrNrB initial_epoch epoch_updaterct||_||_d|_d|_|p|j|_d|_dS)NrF) rrrBepochr_started_default_epoch_updaterrMr)rrBrLrMrs rrzSamplerWrapper.__init__ksS  "  *Id.I*.rr c t||!||j|_||j|_t |jtr@|j ||j t|j|_ nt|jdr|j|jt|j|_ t|jD]@} t!|j #t"$rt%d|jd|dwxYwnzd|_|jr||j|_t|jdr|j|jt|j|_ d|_dS)N set_epochr"r#r$rF)rr%r&r EPOCH_KEYrOr'rBrr( SAMPLER_KEYr*rhasattrrSr+r,r-r.rPrMr/s rr%zSamplerWrapper.resetys  m$$$  $ -d.B CD &t~6DJ$,11  ,,]4;K-LMMM --4<557L**4:666 --t011ATX((oT5Foo78ooo!"D } <!// ;; t|[11 3 &&tz222DL))DH s D&Er1c\d|_t|j}|xjdz c_|S)NTr)rPr,rrr4s rr,zSamplerWrapper.nexts/ DH~~ Q rc|j|j|j|ji}t |jt r!|j||j<|Sr) r&rrTrOr'rBrr7rUr8s rr9zSamplerWrapper.get_statesU  $"3 NDJ&  dlH - - E+/<+B+B+D+DJt' (rrOc |dzSr3)clsrOs rrQz%SamplerWrapper._default_epoch_updaters qyr)rNr)r:r;r<r=r&rTrUr r intr rrrr>rr%r,r9 classmethodrQr?r@s@rrDrD[s8  %OIK 8< / / / / # 45 / / / / / /8DcN#;<a 4S>33[rrDN)typingrrrrrrr r torch.utils.datar torchdata.nodes.base_noder r maprtypesrrrrHrDrZrrrcsQWVVVVVVVVVVVVVVVVVVV$$$$$$11111111 GC4   66666hqk666r A  QR    MMMMMXa[MMMMMr