&`i@ddlZddlmZddlmZmZddlZddlZ ddl m Z m Z ddl mZddlmZddlmZddlmZd Zd Zd ZejeZe Gd d Ze GddeZe GddeZe GddeZe GddeZe GddeZ e GddeZ!e GddeZ"e GddeZ#e d#dej$d e%fd!Z&dej$d ee'fd"Z(dS)$N) OrderedDict)AnyList) OldAPIStackoverride)resize)Repeated)convert_element_to_space_type) TensorType)dceZdZdZddejdefdZdejdedee fdZ de de j fd Zde d e j d e ddfd Zdeddfd Zede fdZedejfdZdS) PreprocessorzDefines an abstract observation preprocessor function. Attributes: shape (List[int]): Shape of the preprocessed output. N obs_spaceoptionscdt|||_|s ddlm}||_n||_|||j|_ttj |j|_ d|_ |j |_dS)Nr)MODEL_DEFAULTS)_legacy_patch_shapes _obs_spaceray.rllib.models.catalogrcopy_options _init_shapeshapeintnpprod_size_isample_obs_for_type_matching)selfrrrs r/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/models/preprocessors.py__init__zPreprocessor.__init__ sY'''# $ ? ? ? ? ? ?*//11DMM#DM%%i?? ,,-- &*o&<&<&>&>###returnct)z&Returns the shape after preprocessing.NotImplementedErrorr&rrs r'rzPreprocessor._init_shape.!!r) observationct)z%Returns the preprocessed observation.r,r&r0s r' transformzPreprocessor.transform2r/r)arrayoffsetcJ||||||jz<dS)z7Alternative to transform for more efficient flattening.N)r3r"r&r0r4r5s r'writezPreprocessor.write6s*.2nn[.I.Ifv **+++r)c |jtzdkr-t|turUt |jt jjr1tj | tj }|j |st||j} |j |sTt!d|t |jt jjr|jnd|jn##t&$r}t!d||d}~wwxYw|xjdz c_dS)z*Checks the shape of the given observation.rz3Observation ({} dtype={}) outside given space ({})!Nz_Observation for a Box/MultiBinary/MultiDiscrete space should be an np.array, not a Python list.)r#OBS_VALIDATION_INTERVALtypelist isinstancergymspacesBoxr r4astypefloat32containsr r% ValueErrorformatdtypeAttributeError)r&r0es r' check_shapezPreprocessor.check_shape:sZ 7, , 1 1K  D((Z..(!h{33::2:FF ?++K88 ;!<  // << $MTT')$/3:>JJ&K--!% O  "    @   1 s0A.D D?)D::D?c|jSN)r"r&s r'sizezPreprocessor.sizeYs zr)ctjdd|jtj}t tttttf}t||r |j |_|S)Ng?rG)r?r@rArr rCDictFlatteningPreprocessorOneHotPreprocessorRepeatedValuesPreprocessorTupleFlatteningPreprocessorAtariRamPreprocessorGenericPixelPreprocessorr>roriginal_space)r&rclassess r'observation_spacezPreprocessor.observation_space]s`JNN4dj NKK  '  & ' $   dG $ $ 7'+I $r)rL)__name__ __module__ __qualname____doc__r?Spacedictr(rrrr r ndarrayr3r8rrJpropertyrNrZr)r'rrsG ? ?#) ?d ? ? ? ?"SY""$s)"""""Z"BJ""""JJBJJJPTJJJJst>cX39Xr)rceZdZdZeedejdede e fdZ eede de jfdZdS) rWzGeneric image preprocessor. Note: for Atari games, use config {"preprocessor_pref": "deepmind"} instead for deepmind-style Atari preprocessing. rrr*c|d|_|d|_|d|_|jr|j|jdf}n|j|jdf}|S)N grayscale zero_meandimr:r)get _grayscale _zero_mean_dim)r&rrrs r'rz$GenericPixelPreprocessor._init_shapewsl!++k22!++k22KK&& ? .Y 1-EEY 1-E r)r0c|||ddddddf}|jdkrt|dd}t||j|j}|jrV|d}|t j}t j||j|jdg}|j r |dz dz }n|d z}|S) z?Downsamples images from (210, 160, 3) by the configured factor.iNT)heightwidthr:rgp?) rJrlrrjmeanrBr rCreshaperk)r&r0scaleds r'r3z"GenericPixelPreprocessor.transforms %%%RVQQQ\* 9r>>F2R888Fty BBB ? C[[^^F]]2:..FZDIq(ABBF ? "slc)FF k !F r)N)r[r\r]r^rrr?r_r`rrrr r rar3rcr)r'rWrWos Xl SY  $s)    XlZBJr)rWceZdZeedejdedee fdZ eede de j fdZdS)rVrrr*cdS)Nrrcr.s r'rz AtariRamPreprocessor._init_shapesvr)r0cb|||ddz dz S)NrCr)rJrBr2s r'r3zAtariRamPreprocessor.transforms4 %%%""9--3s::r)N)r[r\r]rrr?r_r`rrrr r rar3rcr)r'rVrVs XlSY$s)Xl;Z;BJ;;;;;;r)rVc eZdZdZeedejdede e fdZ eede de jfdZeede de jd e dd fd Zd S) rSarOne-hot preprocessor for Discrete and MultiDiscrete spaces. .. testcode:: :skipif: True self.transform(Discrete(3).sample()) .. testoutput:: np.array([0.0, 1.0, 0.0]) .. testcode:: :skipif: True self.transform(MultiDiscrete([2, 3]).sample()) .. testoutput:: np.array([0.0, 1.0, 0.0, 0.0, 1.0]) rrr*ct|tjjr |jjfSt j|jjfSrL) r>r?r@Discreternr sumnvecr.s r'rzOneHotPreprocessor._init_shapes? i!4 5 5 3O%' 'F4?/002 2r)r0c||tjj|j|tjSrL) rJr?r@utilsflattenrrBr rCr2s r'r3zOneHotPreprocessor.transformsC %%%z''EELLRZXXXr)r4r5NcJ||||||jz<dSrL)r3rNr7s r'r8zOneHotPreprocessor.writes*-1^^K-H-Hfv ))***r)r[r\r]r^rrr?r_r`rrrr r rar3r8rcr)r'rSrSs*Xl3SY33$s)3333 XlYZYBJYYYYXlIIBJIIPTIIIIIIr)rSc "eZdZeedejdedee fdZ eede de j fdZeede de j de dd fd Zeeedejfd Zd S) NoPreprocessorrrr*c|jjSrLrrr.s r'rzNoPreprocessor._init_shape $$r)r0c0|||SrL)rJr2s r'r3zNoPreprocessor.transforms %%%r)r4r5Ncptj|d||||jz<dSNF)rr r4ravelr"r7s r'r8zNoPreprocessor.write9.0h{.O.O.O.U.U.W.Wfv **+++r)c|jSrL)rrMs r'rZz NoPreprocessor.observation_spaces r))r[r\r]rrr?r_r`rrrr r rar3r8rbrZrcr)r'rrs# Xl%SY%%$s)%%%%XlZBJXlXXBJXXPTXXXX Xl39Xr)rc &eZdZdZeedejdede e fdZ eede de jfdZeede de jd e dd fd Zeeedejfd Zd S) MultiBinaryPreprocessoraPreprocessor that turns a MultiBinary space into a Box. Note: Before RLModules were introduced, RLlib's ModelCatalogV2 would produce ComplexInputNetworks that treat MultiBinary spaces as Boxes. This preprocessor is needed to get rid of the ComplexInputNetworks and use RLModules instead because RLModules lack the logic to handle MultiBinary or other non-Box spaces. rrr*c|jjSrLrr.s r'rz#MultiBinaryPreprocessor._init_shaperr)r0cj|||tjSrL)rJrBr rCr2s r'r3z!MultiBinaryPreprocessor.transforms. %%%!!"*---r)r4r5Ncptj|d||||jz<dSrrr7s r'r8zMultiBinaryPreprocessor.writerr)ctjdd|jtj}|j|_|S)NgrPrQ)r?r@rArr rCrrX)r&rs r'rZz)MultiBinaryPreprocessor.observation_spaces3JNN3TZrzNJJ #'?  r))r[r\r]r^rrr?r_r`rrrr r rar3r8rbrZrcr)r'rrs/Xl%SY%%$s)%%%%Xl.Z.BJ.... XlXXBJXXPTXXXX Xl39Xr)rc eZdZdZeedejdede e fdZ eede de jfdZeede de jd e dd fd Zd S) rUzPreprocesses each tuple element, then flattens it all into a vector. RLlib models will unpack the flattened output before _build_layers_v2(). rrr*ct|jtjjsJd}g|_t t|jjD]}|jj|}t d |t|}||||j }||j z }n+d}|ttj|jz }|j||fSNrz Creating sub-preprocessor for {})r>rr?r@Tuple preprocessorsrangelenloggerdebugrFget_preprocessorrrNrr r!rappend)r&rrrNispacepreprocessor_class preprocessors r'rz'TupleFlatteningPreprocessor._init_shape s$/3:+;<<<<<s4?12233 4 4AO*1-E LL;BB5II J J J!1%!8!8 !-11%GG  ))# BGEK00111   % %l 3 3 3 3wr)r0c||tj|jtj}|||d|SNrQrrJr zerosrrCr8r&r0r4s r'r3z%TupleFlatteningPreprocessor.transformI %%%2:666 ;q))) r)r4r5Nct|t|jks J|t||jD]&\}}||||||jz }'dSrL)rrzipr8rNr&r0r4r5ops r'r8z!TupleFlatteningPreprocessor.write#s|;3t'9#:#::::K::: T%788  DAq GGAuf % % % af FF  r)rrcr)r'rUrUs XlSY$s)"XlZBJ XlBJPTr)rUc eZdZdZeedejdede e fdZ eede de jfdZeede de jd e dd fd Zd S) rRzPreprocesses each dict value, then flattens it all into a vector. RLlib models will unpack the flattened output before _build_layers_v2(). rrr*ct|jtjjsJd}g|_|jjD]}td |t|}||||j }||j z }n+d}|ttj|jz }|j||fSr)r>rr?r@DictrvaluesrrrFrrrNrr r!rr)r&rrrNrrrs r'rz&DictFlatteningPreprocessor._init_shape2s$/3:?;;;;;_+2244 4 4E LL;BB5II J J J!1%!8!8 !-11%GG  ))# BGEK00111   % %l 3 3 3 3wr)r0c||tj|jtj}|||d|Srrrs r'r3z$DictFlatteningPreprocessor.transformCrr)r4r5Nct|ts.tt|}t |t |jks+Jt |t |jft ||jD]&\}}||||||j z }'dSrL) r>rsorteditemsrrrrr8rNrs r'r8z DictFlatteningPreprocessor.writeJs+{33 C%f[->->-@-@&A&ABBK;3t'9#:#::::    " # #= ::: **,,d.@AA  DAq GGAuf % % % af FF  r)rrcr)r'rRrR+s XlSY$s) XlZBJ Xl  BJ  PT      r)rRc eZdZdZeedejdede e fdZ eede de jfdZeede de jd e dd fd Zd S) rTz0Pads and batches the variable-length list value.rrr*ct|jtsJ|j}t |||j|_d|jj|jzz}|fS)Nr:) r>rr child_spacerrchild_preprocessorrNmax_len)r&rrrrNs r'rz&RepeatedValuesPreprocessor._init_shape[sg$/844444+ "?"2;"?"? # # 4*/)2CCCwr)r0ctj|j}t|tr |D]}|j|n |||d|S)Nr)r rrr>r=rrJr8)r&r0r4elems r'r3z$RepeatedValuesPreprocessor.transformfss$$ k4 ( ( # : :'33D9999 :  ;q))) r)r4r5Nct|ttjfs#t d||t ||jjkr-t d||jjt |||<t|D]6\}}|dz||j j zz}|j |||7dS)Nz&Input for {} must be list type, got {}z$Input {} exceeds max len of space {}r:) r>r=r rarErFrrr enumeraterrNr8)r&r0r4r5rroffset_is r'r8z RepeatedValuesPreprocessor.writeqs+bj'9:: 8??kRR    7 7 76==!8  K((f  -- A AGAtzA(?(D$DDH  # ) )$x @ @ @ @ A Ar)rrcr)r'rTrTWs:: XlSY$s)XlZBJXlAABJAAPTAAAAAAr)rTFrr*ct||j}t|tjjtjjfr t}n%|tkr:t d tt}n|tkr:t d tt}nt|tjjrt }ntt|tjjrt$}nMt|t&rt(}n0t|tjjr |rt,}nt.}|S)z>Returns an appropriate preprocessor class for the given space.aDefaulting to RLlib's GenericPixelPreprocessor because input space has the atari-typical shape {}. Turn this behaviour off by setting `preprocessor_pref=None` or `preprocessor_pref='deepmind'` or disabling the preprocessing API altogether with `_disable_preprocessor_api=True`.aDefaulting to RLlib's AtariRamPreprocessor because input space has the atari-typical shape {}. Turn this behaviour off by setting `preprocessor_pref=None` or `preprocessor_pref='deepmind' or disabling the preprocessing API altogether with `_disable_preprocessor_api=True`.`.)rrr>r?r@r{ MultiDiscreterSATARI_OBS_SHAPErrrFrWATARI_RAM_OBS_SHAPErVrrUrrRr rT MultiBinaryrr)rinclude_multi_binary obs_shapers r'rrsF I%#*-sz/GHII&) o % %  @AG@W@W    0 ) ) )   ((    , E3:+ , , &2 E3:? + +&1 E8 $ $&1 E3:1 2 2&7K&. % r)cDt|dst|tjjrd|_nct|tjjrDg}|jD]&}t|}||'t||_|jS)zAssigns shapes to spaces that don't have shapes. This is only needed for older gym versions that don't set shapes properly for Tuple and Discrete spaces. rrc) hasattrr>r?r@r{rrrrtuple)rshapessrs r'rrs 5' " "( eSZ0 1 1 (EKK sz/ 0 0 (F\ % %,Q// e$$$$--EK ;r))F))logging collectionsrtypingrr gymnasiumr?numpyr ray.rllib.utils.annotationsrrray.rllib.utils.imagesrray.rllib.utils.spaces.repeatedr "ray.rllib.utils.spaces.space_utilsr ray.rllib.utils.typingr rrr; getLoggerr[rrrWrVrSrrrUrRrTr_r<rrrrcr)r'rs?######========))))))444444LLLLLL------  8 $ $ SSSSSSS Sl '''''|'' 'T ;;;;;<;; ; $I$I$I$I$I$I$I $IN \ ( l > $$$$$,$$ $N ((((((( (V )A)A)A)A)A)A)A )AX ''CI'd''' 'T d3ir)