&`i~pddlmZmZmZmZmZddlZddlZ ddl Z ddl m Z ddl mZmZmZddlmZmZddlmZmZmZddlmZeGdd ZdS) )AnyDictListOptionalUnionN)data_equivalence) LARGE_INTEGERone_hotone_hot_multidiscrete)gym_space_from_dictgym_space_to_dict)batchget_base_struct_from_spaceget_dummy_batch_for_space) DeveloperAPIceZdZ d2deeeejfdedee j fdZ ddde fd Z ed Zejd Zed Zdeeeffd ZedeeefddfdZd3dZd3dZd4dZd5deddfdZd3dZ d6dddddddeeeeeefde deede de deedefdZdeedeee fddfd Z!d!Z"d"Z#ddd#d$eeeeeefde ddfd%Z$d&Z%d'Z&d(Z'd7d)Z(d*Z) d8d+Z* d9d,Z+ d8d-efd.Z,d/Z- d6d0Z.d1Z/dS):InfiniteLookbackBufferNrdatalookbackspacec||ng|_t|t|j|_t |jt  |_||_dSN)rminlenr isinstancelist finalizedr)selfrrrs /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/env/utils/infinite_lookback_buffer.py__init__zInfiniteLookbackBuffer.__init__sN !,DD" Hc$)nn55 ' 4888 otherreturnct|toYt|j|jo?|j|jko/|j|jko|j|jko|j|jkS)a\Compares two `InfiniteLookbackBuffers. Args: other: Another object. If another `LookbackBuffer` instance all their attributes are compared. Returns: `True`, if `other` is an `InfiniteLookbackBuffer` instance and all attributes are identical. Otherwise, returns `False`. )rrrrrr space_structrrr"s r__eq__zInfiniteLookbackBuffer.__eq__ s{ u4 5 5 * EJ77 * / *%/1  * !U%77  *  ek) r!c|jSr)_spacers rrzInfiniteLookbackBuffer.space8s {r!c<||_t||_dSr)r)r _space_struct)rvalues rrzInfiniteLookbackBuffer.space<s 7>>r!c|jSr)r,r*s rr%z#InfiniteLookbackBuffer.space_structAs !!r!cd|j|j|j|jrt |jnddS)a?Returns the pickable state of a buffer. The data in the buffer is stored into a dictionary. Note that buffers can also be generated from pickable states (see `InfiniteLookbackBuffer.from_state`) Returns: A dict containing all the data and metadata from the buffer. N)rrrr)rrrrr r*s r get_statez InfiniteLookbackBuffer.get_stateEs<I 6:jJ&tz222d    r!statect}|d|_|d|_|drt|dnd|_|d|_|S)aCreates a new `InfiniteLookbackBuffer` from a state dict. Args: state: The state dict, as returned by `self.get_state`. Returns: A new `InfiniteLookbackBuffer` instance with the data and metadata from the state dict. rrrNr)rrrr rr)r1buffers r from_statez!InfiniteLookbackBuffer.from_stateVs[()) + ->CGnV*5>:::RV Fm  r!c|jr#tjd|j||_dS|j|dS)z1Appends the given item to the end of this buffer.c4tj||ggdSNr)axisnp concatenatedis rz/InfiniteLookbackBuffer.append..nsR^QH1===r!N)rtree map_structurerappendritems rrBzInfiniteLookbackBuffer.appendjsN > #*==ty$DIII I  T " " " " "r!c|jrLtjd|jt |t rt j|n||_dS|D]}||dS)z7Appends all items in `items` to the end of this buffer.c2tj||gdSr7r9r<s rr?z/InfiniteLookbackBuffer.extend..zsR^QF;;;r!N) rr@rArrrr:arrayrB)ritemsrDs rextendzInfiniteLookbackBuffer.extendss > "*;; #-eT#:#:E DIII " " D!!!! " "r!c^|j|dS)zConcatenates the data of `other` (w/o its lookback) to `self`. Args: other: The other InfiniteLookbackBuffer to be concatenated to self. N)rrIgetr&s rconcatzInfiniteLookbackBuffer.concats( %%%%%r!indexc|jr$tjfd|j|_dS|jdS)zRemoves the item at `index` from this buffer, but does NOT return it. Args: index: The index to pop out of this buffer (w/o returning it from this method). c2tj|dSr7)r:delete)srNs rr?z,InfiniteLookbackBuffer.pop..s")Au1555r!N)rr@rArpop)rrNs `rrSzInfiniteLookbackBuffer.popsS > !*5555tyDIII IMM% r!cX|js"t|j|_d|_dSdS)zFinalizes this buffer by converting internal data lists into numpy arrays. Thereby, if the individual items in the list are nested structures, the resulting buffer content will be a nested struct of np.ndarrays (leafs). TN)rrrr*s rfinalizezInfiniteLookbackBuffer.finalizes4 ~ "di((DI!DNNN " "r!F)neg_index_as_lookbackfillone_hot_discrete_ignore_last_ts_add_last_ts_valueindicesrVrWrXrYrZc|}nt|tr|}nqt|tr*fd|D}jrt |}n2t|tsJ|}|S)a Returns data, based on the given args, from this buffer. Args: indices: A single int is interpreted as an index, from which to return the individual data stored at this index. A list of ints is interpreted as a list of indices from which to gather individual data in a batch of size len(indices). A slice object is interpreted as a range of data to be returned. Thereby, negative indices by default are interpreted as "before the end" unless the `neg_index_as_lookback=True` option is used, in which case negative indices are interpreted as "before ts=0", meaning going back into the lookback buffer. neg_index_as_lookback: If True, negative values in `indices` are interpreted as "before ts=0", meaning going back into the lookback buffer. For example, a buffer with data [4, 5, 6, 7, 8, 9], where [4, 5, 6] is the lookback buffer range (ts=0 item is 7), will respond to `get(-1, neg_index_as_lookback=True)` with `6` and to `get(slice(-2, 1), neg_index_as_lookback=True)` with `[5, 6, 7]`. fill: An optional float value to use for filling up the returned results at the boundaries. This filling only happens if the requested index range's start/stop boundaries exceed the buffer's boundaries (including the lookback buffer on the left side). This comes in very handy, if users don't want to worry about reaching such boundaries and want to zero-pad. For example, a buffer with data [10, 11, 12, 13, 14] and lookback buffer size of 2 (meaning `10` and `11` are part of the lookback buffer) will respond to `get(slice(-7, -2), fill=0.0)` with `[0.0, 0.0, 10, 11, 12]`. one_hot_discrete: If True, will return one-hot vectors (instead of int-values) for those sub-components of a (possibly complex) space that are Discrete or MultiDiscrete. Note that if `fill=0` and the requested `indices` are out of the range of our data, the returned one-hot vectors will actually be zero-hot (all slots zero). _ignore_last_ts: Whether to ignore the last record in our internal `self.data` when getting the provided indices. _add_last_ts_value: Whether to add the value of this arg to the end of the internal `self.data` buffer (just for the duration of this get operation, not permanently). N)rXrYrWrVrXrYrZc Fg|]}|S)r])_get_int_index).0idxrZrYrWrVrXrs r z.InfiniteLookbackBuffer.get..sR   ##*?%5$3'9 $   r!) _get_all_datarslice _get_slicerrrintr_)rr[rVrWrXrYrZrs` ````` rrKzInfiniteLookbackBuffer.gets*` ?%%!1 /&DD ' ' ??&;!1 /#5 #DD & &          #   D~ #T{{gs++ + ++&&&;!1 /#5 'D r!cx|jr%tdt|jdt t t tfrfd|jD}n.sG$.a#ug1F#G#GNQYYQr!N)rrr) r RuntimeErrortype__name__rrfrkrlrrrr)rr"rs ` r__add__zInfiniteLookbackBuffer.__add__s" > Td4jj>QTTTUU U%#ug!677 )!YE#9:: )y5:en.>.>#??y5()j r!c,||S)z0Support squared bracket syntax, e.g. buffer[:5].)rKrCs r __getitem__z"InfiniteLookbackBuffer.__getitem__'sxx~~r!c4|||dS)N)new_data at_indices)set)rkeyr-s r __setitem__z"InfiniteLookbackBuffer.__setitem__+s %C00000r!)rurVruc|||dSt|tr||||dSt|tr5t |D]#\}}|||||$dSt|tsJ||||dS)aOverwrites all or some of the data in this buffer with the provided data. Args: new_data: The new data to overwrite existing records with. at_indices: A single int is interpreted as an index, at which to overwrite the individual record stored at this index with `new_data`. A list of ints is interpreted as a list of indices, which to overwrite with `new_data`, which must be a batch of size `len(at_indices)`. A slice object is interpreted as a range, which to overwrite with `new_data`. Thereby, negative indices by default are interpreted as "before the end" unless the `neg_index_as_lookback=True` option is used, in which case negative indices are interpreted as "before ts=0", meaning going back into the lookback buffer. neg_index_as_lookback: If True, negative values in `at_indices` are interpreted as "before ts=0", meaning going back into the lookback buffer. For example, a buffer with data [4, 5, 6, 7, 8, 9], where [4, 5, 6] is the lookback buffer range (ts=0 item is 7), will handle a call `set(99, at_indices=-1, neg_index_as_lookback=True)` with `6` being replaced by 99 and to `set([98, 99, 100], at_indices=slice(-2, 1), neg_index_as_lookback=True)` with `[5, 6, 7]` being replaced by `[98, 99, 100]`. N)slice_rV)rarV) _set_all_datarrd _set_slicer enumerate_set_int_indexrf)rrtrurVr>ras rrvzInfiniteLookbackBuffer.set.s<     x ( ( ( ( (  E * *  OO!&;        D ) ) #J//  3##QK*?$  j#.. . ..   &;      r!cZ|}t||jz dS)z=Return the length of our data, excluding the lookback buffer.r)len_incl_lookbackmaxr)rlen_s r__len__zInfiniteLookbackBuffer.__len__ds*%%''4$-'+++r!c|jr,ttj|jdSt|jSNr)rrr@flattenrr*s rrz(InfiniteLookbackBuffer.len_incl_lookbackjs9 > "t|DI..q122 2ty>> !r!c t|jd|jd|jd|jd|j|jddS)N(z <- lookback(z) | ))rnrorrr*s r__repr__zInfiniteLookbackBuffer.__repr__psmDzz" H HTY~ ~%> H H  H H+/9T]^^+D H H H r!c^|d|sdnd}|r|||j}|S)NrMr%)_one_hotr%)rrXrYrs rrcz$InfiniteLookbackBuffer._get_all_datavsA;?:tt;<  G==D4E=FFD r!cN||tdddSr)r|rd)rrts rr{z$InfiniteLookbackBuffer._set_all_data|s$ %4..11111r!c |j}|r2|jrtjd|j}n|jdd}|X|jr*tjd||}n't j||}|||t|duzt|z \}} } d} |dkrF|jrtjfd|} n|} |r| | |j } || dks| dkr|jr| rT|j t j |g| z} nt|j || |} | tjd | | } n| } | rT|j t j |g| z} nt|j || |} | tjd | | } n9| } n6|j |g} nt|j |d|g} | | z| | ngz| | zz} | '|jrtjfd |S|S| S) Nc|ddSNrMrjrRs rr?z3InfiniteLookbackBuffer._get_slice..1SbS6r!rMc,tj||Srr:rB)rRts rr?z3InfiniteLookbackBuffer._get_slice..s1ar!)len_self_plus_lookbackrc|SrrjrRrzs rr?z3InfiniteLookbackBuffer._get_slice..s !F)r!r fill_value batch_sizerXc.tj||gSrr9s0rRs rr?z3InfiniteLookbackBuffer._get_slice..s"."a*A*Ar!c.tj||gSrr9rs rr?z3InfiniteLookbackBuffer._get_slice..s".!R*A*Ar!c|Srrjrs rr?z3InfiniteLookbackBuffer._get_slice..s AfIr!)rrr@rAcopyr:rB_interpret_slicerrfrr%rrGr) rrzrWrVrXrYrZ data_to_use slice_lenfill_left_countfill_right_count data_slice fill_batchs ` rrez!InfiniteLookbackBuffer._get_slices0i  -~ -"01A1A49MM "in  )~ P"000$$&&& !i (8(8(:(: J'+'6-= &&& "-%)%7AA&&&& &0 #0z)%'Xtf7G.G%H%H %> J'+'7-= &&& "-%)%7AA&&&& &0 :%"&JJ2 J'+'(-= "J0%/%;zzE #334  ~ +)*=*=*=*={KKK"6**r!cL|\}}} jr#fd}tj|j|dSt jt |ksJ|j<dS#t $rtdd|dwxYw)Ncjr"j|dsJt|t|ksJ||<dSr)rcontainsr)rRnrrzs r__setz0InfiniteLookbackBuffer._set_slice..__sets[z9#z221Q488888qy>>SVV3333 !AfIIIr!z$Cannot `set()` value via at_indices= (option neg_index_as_lookback=z;)! Slice of data does NOT have the same size as `new_data`.)rrr@rArrAssertionError IndexError)rrtrzrV__InfiniteLookbackBuffer__sets` ` rr|z!InfiniteLookbackBuffer._set_slices //8MNN1a ~ -"""""" "5$)X>>>>>49V,--X>>>>$, &!!!   =v==)>===  s(B 7B!B#rac|j}|r2|jrtjd|j}n|jdd}|H|jrtjd||}n)|}||dks|r |jz|r"dkrt||jz|duz  |jrtjfd|}n|}nA#t$r4} |+|j |cYd} ~ St|j |d|cYd} ~ S| d} ~ wwxYw|r| ||j }|S)Nc|ddSrrjrs rr?z7InfiniteLookbackBuffer._get_int_index..rr!rMc,tj||Srr)rRlasts rr?z7InfiniteLookbackBuffer._get_int_index..sBIa$6$6r!rTc|Srrj)rRras rr?z7InfiniteLookbackBuffer._get_int_index..1s AcFr!r) rrr@rArrBrrrrrrr%) rrarWrVrXrYrZrres ` rr_z%InfiniteLookbackBuffer._get_int_index si  -~ -"01A1A49MM "in  )~ 7"066 EW *..00 ""#5666 !88,8-#%C ! HS1WWd))dm+$/FGC ~ ()*:*:*:*:KHH"3'   :%KKKKKK0J# %5    :==t'899D s0>(C'' D%1 D ;D%D D%D  D%c"|dks|r jz|rdkrtjz jr#fd}tj|j|dS|j<dS#t $rt d|d|dwxYw)Nrchjr$j|s J|||<dSr)rr)rRr actual_idxrs rrz4InfiniteLookbackBuffer._set_int_index..__setWs>z9#z2215588q885$%AjMMMr!zCannot `set()` value at index rz)! Out of range of buffer data.)rrrr@rArr)rrtrarVrrs` @rr~z%InfiniteLookbackBuffer._set_int_indexGs  ??3?3J ! 3Z!^^TT]2J ~ 1&&&&&& "5$)X>>>>>(0 *%%%   ###)>###  s(A-! A--!Bc|t||jz}|j}|j}||j}n#|dkr|r |j|z}n||z}n |j|z}||}n#|dkr|r |j|z}n||z}n |j|z}dx}}|dkr|dkrt ||z }d}dx}}nf||kr||krt ||z }d}|x}}nA|dkr| }d}n5||kr||z }|}n'|dkr!||kr ||z dz}|dz }| dz }t }|dkr|dks|t ks J||f|j|jnd}t|||}td||z ||dkrdndz z|z} || ||fS)NrrM) rrstartstopabsr steprdr) rrzrVrrrrrrrs rrz'InfiniteLookbackBuffer._interpret_slicefsG " )%(YY%> " { =MEE QYY% 7 -/6ME)E <)DD AXX% 5}t+.4=4'D-..* 199!%$,//O  EDD , , ,9O1O1O"54<00 O1 1EDD QYY$fOEE + + +#&<< )DD AXX..."'*@"@1"D.2 $uqy !>DzztqyyD]N,B,B,BeT],B,BC$k5v{{1udD))D5LDAAr,JKPTTUU y/3CCCr!c%tdt|jddt|trfd|D}nt j|}|S)NzCannot `one_hot` data in `z6` if a gym.Space was NOT provided during construction!ct|tjjrt ||jSt|tjjrt||jS|S)N)depth)depths) rgymspacesDiscreter r MultiDiscreter nvec)dat_rs r_convertz1InfiniteLookbackBuffer._one_hot.._convertsa%!455 Ft573333E3:#;<< F,T%*EEEEKr!c<g|]}tj|Srj)r@rA)r`dslicerr%s rrbz3InfiniteLookbackBuffer._one_hot..s5GM"8V\BBr!) ValueErrorrnrorrr@rA)rrr%rs `@rrzInfiniteLookbackBuffer._one_hots  BT$ZZ-@BBB      dD ! ! DQUDD%hlCCD r!)NrN)r#N)r"rr#N)rMr)FF)NFFFN)F)0ro __module__ __qualname__rrrr:ndarrayrfrSpacer boolr'propertyrsetterr%rstrrr0 staticmethodr4rBrIrLrSrUrdrKrkrlrprrrxrvrrrrcr{rer|r_r~rrrjr!rrrs37%)  uT2:-./   "     '      0X \??\?""X" 4S>    "$sCx.-E\&####"""" &&&& ! ! !d ! ! ! !"""";?W',"!& %,0WWW%UDI 567W $ W sm W  WW%SMW WWWWr)4!93wNO) !))))V111>B&+ 444U3tCy#89: 4 $ 4  4444l,,, """     222 #mmmmf$ B#99 9999vF $ ODODODODbr!r)typingrrrrr gymnasiumrnumpyr:r@gymnasium.utils.env_checkerrray.rllib.utils.numpyr r r ray.rllib.utils.serializationr r "ray.rllib.utils.spaces.space_utilsrrrray.util.annotationsrrrjr!rrs&33333333333333 888888OOOOOOOOOOPPPPPPPP .-----x x x x x x x x x x r!