&`i-3ddlZddlmZddlmZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZdd lmZmZmZdd lmZdd lmZdd lmZdd lmZejeZedGdde ZdS)N)Path)List)AlgorithmConfig)Columns) EnvRunner)SingleAgentEnvRunner)SingleAgentEpisode)OverrideToImplementCustomLogic5OverrideToImplementCustomLogic_CallToSuperRecommendedoverride)pack_if_needed) EpisodeType) PublicAPI)log_oncealpha) stabilityc0eZdZdZeeedeffd Zeee ddddddde de d e d e d e d e e f fd Zeee ddZe de ed dfdZxZS)OfflineSingleAgentEnvRunnerz7The environment runner to record the single agent case.configc rtjdd|i|d|_tjj}|jj |j |j_|j j |_ |j j |_ |j j|_|j j|_d|_|j j|_|j j|_d|jdz|_|jdkrddl}|jdi|j|_no|jdkrdd lm}|jdi|j|_nF|jd krddl }|j!di|j|_n|jtEd |jd |j #d |ji|j j$|_$|j j%|_%|j j&|_&|j&rd|_'nd|_'|j j(|_)d|_*g|_+dS)NrF)cpuzrun-gcsrs3)fsabszUnknown filesystem: z9. Filesystems can be 'gcs' for GCS, 's3' for S3, or 'abs' filesystemT),super__init__episodes_to_numpyraydata DataContext get_currentexecution_optionsresource_limitscopynum_cpus_per_env_runnerroutput_write_methodoutput_write_method_kwargsoutput_filesystemroutput_filesystem_kwargsfilesystem_kwargsfilesystem_objectoutput output_pathenvlower subdir_path worker_indexzfill worker_pathgcsfs GCSFileSystempyarrowr S3FileSystemadlfsAzureBlobFileSystem ValueErrorupdateoutput_write_episodesoutput_compress_columnsoutput_max_rows_per_filewrite_data_this_iteroutput_write_remaining_datawrite_remaining_data_sample_counter_samples)selfrkwargs data_contextr8rr< __class__s x/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/offline/offline_env_runner.pyr z$OfflineSingleAgentEnvRunner.__init__s 111&111"'x+7799   * : ? ?2 @   &6$(;#B *.+*P'+7!%!E!% ;-;?0022"t'8$:$@$@$C$CC ?e # # LLL%8U%8%R%R4;Q%R%RD " " _ $ $ " " " " " "%4R_%N%Nt7M%N%ND " " _ % % LLL%>U%>%X%XAW%X%XD " " _ (7t777  '..d4    &*[%F"'+{'J$)- (L%  ( -(-D % %)-D %%)K$K! ! NF num_timesteps num_episodesexplorerandom_actions force_resetrOrPrQrRrSreturnc t|||||}|xjdz c_|jrtddl ddl t drtdtd|DsJ|j fd|Dn| ||j r$t|j |j krd |_|jr|j rd |_t|j |j kri|j d|j }|j |j d|_ t j|}t|j |j kin$t j|j } t'|j|j|jd |jd z} t3||j| fi|jtd | dn1#t:$r$} t| Yd} ~ nd} ~ wwxYw|j dt|j |S)z2Samples from environments and writes data to disk.rNrNmsgpackzPacking episodes with `msgpack` and encode array with `msgpack_numpy` for serialization. This is needed for recording episodes.c3(K|] }|jduVdS)FN)is_numpy).0epss rL z5OfflineSingleAgentEnvRunner.sample..s)@@s|u,@@@@@@rMclg|]0}|j1S))default)packb get_stateencode)rZr[mnprWs rL z6OfflineSingleAgentEnvRunner.sample..s3WWWs}} CCWWWrMTF-rzWrote samples to storage at .recording_buffer_size)keyvalue)!rsamplerFr@rW msgpack_numpyrloggerinfoallrGextend_map_episodes_to_datarBlenrCr"r# from_itemsrr1joinpathr4r7r6getattrr*as_posixr+ Exceptionerrormetrics log_value)rHrOrPrQrRrSsamplessamples_to_write samples_dspatherbrWrKs @@rLriz"OfflineSingleAgentEnvRunner.sample{s''..'%)# !   !  % 0 NNN ' ' ' ' ""  *@@@@@@@ @ @@ M WWWWWwWWW      & &w / / /  ( 14=!!T%BBB,0)  $# , @,1)$-((D,III(,}5Tt7T5T'U$%)M$2O2Q2Q$RDM!$!4!45E!F!FJ$-((D,III!X00??  )**Xd.//Xd.1KT5I1K1K1Q1QRS1T1TTUU > D$<==MMOO'+'F B4BBBCCCC    Q  'dm$$    s7B3I++ J5JJc|jr"|jrtj|j}|xjdz c_ t |j|j |j d|j dz}t||j |fi|jt d|dn1#t$$r$}t |Yd}~nd}~wwxYwt dt+|jdS)zWrites the reamining samples to disk Note, if the user defined `max_rows_per_file` the number of rows for the remaining samples could be less than the defined maximum row number by the user. rVrdrz"Wrote final samples to storage at zY. Note Note, final samples could be smaller in size than `max_rows_per_file`, if defined.NzExperience buffer length: )rGrEr"r#rqrFrr1rrr4r7r6rsr*rtr+rkrlrurvdebugrp)rHr{r|r}s rLstopz OfflineSingleAgentEnvRunner.stopsu = T6 ,,T];;J  A %  )**Xd.//Xd.1KT5I1K1K1Q1QRS1T1TTUU > D$<==MMOO'+'F 8888     Q   F#dm2D2DFFGGGGGsB3C:: D(D##D(ryc.|D] ttD]tjjtjjtjjtj tj j vr"t n tj tj j vr"tntjtjtj j vr%t dzn dztjtdz krdnjtjtdz krdnji fdjD}j|dS)zConverts list of episodes to list of single dict experiences. Note, this method also appends all sampled experiences to the buffer. Args: samples: List of episodes to be converted. rVFc i|]E}||jvr#t|n|FSr)rAr get_extra_model_outputs)rZkirirHs rL zEOfflineSingleAgentEnvRunner._map_episodes_to_data..(sl  <<<*&*H*HA*N*NOOO#;;AqAA rMN)rangerprEPS_IDid_AGENT_IDagent_id MODULE_ID module_idOBSrAr get_observationsACTIONS get_actionsREWARDS get_rewardsNEXT_OBS TERMINATEDS is_terminated TRUNCATEDS is_truncatedextra_model_outputskeysrGappend)rHry sample_datarris` @@rLroz1OfflineSingleAgentEnvRunner._map_episodes_to_datas# 2# 2F3v;;''! 2! 2NFJ$fo%v'7K{d&BBB"00G0G0J0J!K!K!K0033O$*FFF&4F4F4Fq4I4I%J%J%J++A..OV%7%7%:%:${d&BBB'5V5L5LQQRU5S5S&T&T&T00Q77'3v;;?***/-&3v;;?**).,-. "(!;!@!@!B!B / @ $$[1111C! 2# 2# 2rM)rTN)__name__ __module__ __qualname____doc__r rr rr r intboolrr rirrrro __classcell__)rKs@rLrrsAA X"##:X/XXXXX;:$#XtX"###" $!]]]] ]  ]  ]]  !]]]]]$#$#]~Xi##H#H#H$##HJ$-2T+->-24-2-2-2$#-2-2-2-2-2rMr) loggingpathlibrtypingrr"%ray.rllib.algorithms.algorithm_configrray.rllib.core.columnsrray.rllib.env.env_runnerr%ray.rllib.env.single_agent_env_runnerr"ray.rllib.env.single_agent_episoder ray.rllib.utils.annotationsr r r ray.rllib.utils.compressionr ray.rllib.utils.typingrray.util.annotationsrray.util.debugrLogger__file__rkrrrMrLrs{ AAAAAA******......FFFFFFAAAAAA 766666......******######  ! !  WU2U2U2U2U2"6U2U2U2U2U2rM