&`iH. ddlmZmZmZmZmZmZmZmZm Z m Z m Z m Z ddl ZddlmZerVddlmZddlZddlZddlZddlmZddlmZddlmZddlmZddl m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'dd l(m)Z)dd l*m+Z+m,Z,ddl-m.Z.e dZ/ e e/e0e1fZ2 e e e3dfee3fZ4e dZ5 e e6ddfZ7 e dZ8 ee6efZ9 e0Z: e0Z; e0Z< ee e3e e3e e3e3ffe e3e e3e3fffZ= e ee6e ee>e6ffe>e6fZ? e0Z@ e e3e6fZA e eejBfZC edgeeCfZD eZE e6ZF e6ZG eeFdfZH e dZI eeFedgeJfZK eeEeIgeGfZL e eeGeeGedgeJffZM ee6e2fZN e e dZO e e3e6fZP e3ZQ eeEefZR eeAeRfZS eZT eZU e0ZV eZW ee6dfZX eZY e dZZ eZZ[ e dZ\ eZ] ee]e\fZ^ ee\Z_ ee6e\fZ` e eaeee e3eafee e3e e3eafffZb e0Zc e0Zd e ee e/e/fee/fZe e0Zf ee6e/fZg e ddee6effZh e ejijjee6ejijjfe ejijjdffZk eeeZl e e2elefZm eGddZneGd d!ZoeGd"d#Zpe d$ZqdS)%) TYPE_CHECKINGAnyCallableDictHashableListOptionalSequenceTupleTypeTypeVarUnionN) OldAPIStack)NDArray)MultiRLModuleSpec) RLModuleSpec) EnvContext)MultiAgentEpisode)SingleAgentEpisode)DynamicTFPolicyV2)EagerTFPolicyV2) PolicySpec)MultiAgentBatch SampleBatch)ViewRequirement)z NDArray[Any]z jnp.ndarrayz tf.Tensor torch.Tensor.)ztorch.nn.Modulez keras.Modelz torch.deviceint)rrrr)rrr)rrr)ztorch.optim.Optimizerzkeras.optimizers.Optimizer)rz tf.Variablerc&eZdZdZdededefdZdS)AgentConnectorDataTypeaData type that is fed into and yielded from agent connectors. Args: env_id: ID of the environment. agent_id: ID to help identify the agent from which the data is received. data: A payload (``data``). With RLlib's default sampler, the payload is a dictionary of arbitrary data columns (obs, rewards, terminateds, truncateds, etc). env_idagent_iddatac0||_||_||_dSN)r r!r")selfr r!r"s j/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/utils/typing.py__init__zAgentConnectorDataType.__init__5s    N)__name__ __module__ __qualname____doc__strrr'r(r&rr)sGscr(rc*eZdZdZdedededefdZdS)ActionConnectorDataTypea?Data type that is fed into and yielded from agent connectors. Args: env_id: ID of the environment. agent_id: ID to help identify the agent from which the data is received. input_dict: Input data that was passed into the policy. Sometimes output must be adapted based on the input, for example action masking. So the entire input data structure is provided here. output: An object of PolicyOutputType. It is is composed of the action output, the internal state output, and additional data fetches. r r! input_dictoutputc>||_||_||_||_dSr$)r r!r1r2)r%r r!r1r2s r&r'z ActionConnectorDataType.__init__Ns$   $ r(N)r)r*r+r,r-TensorStructTypePolicyOutputTyper'r.r(r&r0r0?sX     %  !      r(r0c2eZdZdZdeeefddfdZdS)AgentConnectorsOutputa=Final output data type of agent connectors. Args are populated depending on the AgentConnector settings. The branching happens in ViewRequirementAgentConnector. Args: raw_dict: The raw input dictionary that sampler can use to build episodes and training batches. This raw dict also gets passed into ActionConnectors in case it contains data useful for action adaptation (e.g. action masks). sample_batch: The SampleBatch that can be immediately used for querying the policy for next action. raw_dict sample_batchrc"||_||_dSr$)r8r9)r%r8r9s r&r'zAgentConnectorsOutput.__init__os! (r(N)r)r*r+r,rr-r4r'r.r(r&r7r7_sM  )S"223)CP))))))r(r7T)rtypingrrrrrrr r r r r r gymnasiumgymray.rllib.utils.annotationsr jax.numpynumpyjnpkeras tensorflowtftorch numpy.typingr(ray.rllib.core.rl_module.multi_rl_moduler"ray.rllib.core.rl_module.rl_modulerray.rllib.env.env_contextr!ray.rllib.env.multi_agent_episoder"ray.rllib.env.single_agent_episoder%ray.rllib.policy.dynamic_tf_policy_v2r#ray.rllib.policy.eager_tf_policy_v2rray.rllib.policy.policyrray.rllib.policy.sample_batchrr!ray.rllib.policy.view_requirementr TensorTypedicttupler4r TensorShape NetworkTyper- DeviceTypeRLModuleSpecType StateDictAlgorithmConfigDictPartialAlgorithmConfigDictModelConfigDictConvFilterSpectypeFromConfigSpec EnvConfigDictEnvIDEnvEnvType EnvCreatorAgentIDPolicyIDModuleIDMultiAgentPolicyConfigDict EpisodeTypeboolIsPolicyToTrainAgentToModuleMappingFnShouldModuleBeUpdatedFn PolicyStateTFPolicyV2Type EpisodeIDUnrollIDMultiAgentDict MultiEnvDict EnvObsType EnvActionType EnvInfoDictFileTypeViewRequirementsDict ResultDictLocalOptimizer OptimizerParamParamRef ParamDict ParamListNamedParamDictfloatLearningRateOrSchedule GradInfoDictLearnerStatsDictModelGradients ModelWeightsModelInputDictSampleBatchTypespacesSpace SpaceStruct StateBatchesr5rr0r7r;r.r(r&rs                            333333BLLLLLL$$$$$$JJJJJJ??????444444CCCCCCEEEEEEGGGGGGCCCCCC222222JJJJJJJJAAAAAAM N  T501NE#s(OT#Y./ 45  3- .  <=5 cN M"   #uS%S/)*E#uS#X2F,G GHtCsD#~!667sBC   c3h SW  |nhw&77 8  D B D!(L"89@=> CHh/@&ABDHI"!7K"8("BC5 X h!23 4d :;= 3(()  eBCD8 #s(O 7 Ogsl#9E>)*   B 9  +C!223  LM+  % +,8 2 5 ! > K c5j!9 eCJ !sE#u*%% &')     tE*j"89:D5   *  > ))))))) )4 GCLLr(