&`i2>ddlZddlmZmZddlmZmZmZmZm Z m Z m Z ddl Z ddlZddlZddlmZmZddlmZddlmZmZddlmZddlmZdd lmZdd lm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&erdd l'm(Z(dd l)m*Z*dd l+m,Z,ddl-m.Z.ddl/m0Z0ej1e2Z3e\Z4Z5Z6eddddde e7de8de e7fdZ9ede&de&de&fdZ:e d[de%de e$de;de&fdZd#e ed$e e8de;d%e;dd&f d'Z?ed(e d)d*e d+e#fde e d,fd-Z@ed\d/e&d0e7de&fd1ZAed/e&de&fd2ZBe d]d3e d4d5e;defd6ZCe d^d8e d9e&d:ed;dde&fd>ZEed_d/e&d?e eFde&fd@ZGe d]dAe e8dBfdCe;ded;fdDZHed`dFZIedadGZJe dbd#dEdKeFdLe7dMe7fdNZKedOe"dPe"dQe7ddfdRZLedSe%de&fdTZMedUe d)dVe&ddfdWZNdXZOeGdYdZZPdS)cN) OrderedDictdeque) TYPE_CHECKINGAnyCallableListOptionalTypeUnion)Discrete MultiDiscrete) force_list) DeveloperAPI PublicAPI) try_import_tf) SMALL_NUMBER)get_base_struct_from_space)LocalOptimizerModelGradients NetworkTypePartialAlgorithmConfigDict SpaceStructTensorStructType TensorType)AlgorithmConfig) ParamDict EagerTFPolicyEagerTFPolicyV2TFPolicy) grad_clipgradients_dictrr# grad_clip_byreturncb|dS|dkrN|D]%\}}t|| |||<&dS|dkrL|D]#\}}t||||<$dS|dksJtt ||\}}t| |D] \}}|||< |S)a?Performs gradient clipping on a grad-dict based on a clip value and clip mode. Changes the provided gradient dict in place. Args: gradients_dict: The gradients dict, mapping str to gradient tensors. grad_clip: The value to clip with. The way gradients are clipped is defined by the `grad_clip_by` arg (see below). grad_clip_by: One of 'value', 'norm', or 'global_norm'. Returns: If `grad_clip_by`="global_norm" and `grad_clip` is not None, returns the global norm of all tensors, otherwise returns None. Nvaluenorm global_norm) copyitemstf clip_by_value clip_by_normclip_by_global_normlistvalueszipkeys)r$r#r%kv clipped_gradsr*s l/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/utils/tf_utils.pyclip_gradientsr9$se,w"''))//11 K KDAq " 0 0YJ J JN1   K K   "''))//11 > >DAq "9 = =N1   > > },,,,%'%;%; &&(( ) )9& & " {++--2244mDD " "DAq !N1  ypredctj|dg\}}tj||z dg\}}tdd||tzz z S)a,Computes the explained variance for a pair of labels and predictions. The formula used is: max(-1.0, 1.0 - (std(y - pred)^2 / std(y)^2)) Args: y: The labels. pred: The predictions. Returns: The explained variance given a pair of labels and predictions. r)axesg)r-nnmomentsmaximumr)r;r<_y_vardiff_vars r8explained_variancerFUseu}}QaS}))HAu%--Ds-33KAx ::dAU\-A!BC D DDr:Finputs spaces_struct time_axisc tj|}|tj|ndgt|z}d}d}g}t||D]\}} t|}t|} || d}|r| d}t| trh|rt |||zg}| t t|| tj t| trj|r t |||zdg}| t t|| tj J|r!t |||zdg}nt ||dg}| t |tj t|d} |rt | ||dg} | S)aKFlattens arbitrary input structs according to the given spaces struct. Returns a single 1D tensor resulting from the different input components' values. Thereby: - Boxes (any shape) get flattened to (B, [T]?, -1). Note that image boxes are not treated differently from other types of Boxes and get flattened as well. - Discrete (int) values are one-hot'd, e.g. a batch of [1, 0, 3] (B=3 with Discrete(4) space) results in [[0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1]]. - MultiDiscrete values are multi-one-hot'd, e.g. a batch of [[0, 2], [1, 4]] (B=2 with MultiDiscrete([2, 5]) space) results in [[1, 0, 0, 0, 1, 0, 0], [0, 1, 0, 0, 0, 0, 1]]. Args: inputs: The inputs to be flattened. spaces_struct: The structure of the spaces that behind the input time_axis: Whether all inputs have a time-axis (after the batch axis). If True, will keep not only the batch axis (0th), but the time axis (1st) as-is and flatten everything from the 2nd axis up. Returns: A single 1D tensor resulting from concatenating all flattened/one-hot'd input components. Depending on the time_axis flag, the shape is (B, n) or (B, T, n). .. testcode:: :skipif: True # B=2 from ray.rllib.utils.tf_utils import flatten_inputs_to_1d_tensor from gymnasium.spaces import Discrete, Box out = flatten_inputs_to_1d_tensor( {"a": [1, 0], "b": [[[0.0], [0.1]], [1.0], [1.1]]}, spaces_struct=dict(a=Discrete(2), b=Box(shape=(2, 1))) ) print(out) # B=2; T=2 out = flatten_inputs_to_1d_tensor( ([[1, 0], [0, 1]], [[[0.0, 0.1], [1.0, 1.1]], [[2.0, 2.1], [3.0, 3.1]]]), spaces_struct=tuple([Discrete(2), Box(shape=(2, ))]), time_axis=True ) print(out) .. testoutput:: [[0.0, 1.0, 0.0, 0.1], [1.0, 0.0, 1.0, 1.1]] # B=2 n=4 [[[0.0, 1.0, 0.0, 0.1], [1.0, 0.0, 1.0, 1.1]], [[1.0, 0.0, 2.0, 2.1], [0.0, 1.0, 3.0, 3.1]]] # B=2 T=2 n=4 Nrr?axis)treeflattenlenr3r-convert_to_tensorshape isinstancer reshapeappendcastone_hotfloat32r concat) rGrHrI flat_inputs flat_spacesBToutinput_spacerRmergeds r8flatten_inputs_to_1d_tensorrbhsz,v&&K  $ ]###Vc+&& & A A C[+6644 %%f--   9aA !H eX & & 4 5FQUG44 JJrwwwvu55rzBB C C C C } - - 4 9FQUBK88 JJrwwwvu55rzBB C C C C 5FQUBK88FQG44 JJrwwvrz22 3 3 3 3 YYsY $ $F0FQ2J// Mr:c tdkrddlm}|}nS tj}n3#t$r&tjj}YnwxYwd|DS)zReturns a list of GPU device names, e.g. ["/gpu:0", "/gpu:1"]. Supports both tf1.x and tf2.x. Returns: List of GPU device names (str). r?r) device_libc.g|]}d|jv |jS)GPU) device_typename).0ds r8 z#get_gpu_devices..s& > > >qu '='=AF'='='=r:) tfvtensorflow.python.clientrdlist_local_devicesr-configlist_physical_devices Exception experimental)rddevicess r8get_gpu_devicesrts axx777777//11 Ei5577GG E E Ei,BBDDGGG E ? >G > > >>sA-A76A7T)r`r(rhrIrOr`r(rhrOztf1.placeholdercddlm}|t|tjjtjjfr=|r||dStj fdt|St d|rdndz|j z|jtjkr t"jn|jS|J|j dd}t d|rdndzt|t&r|n t'|z|jtjkr t"jn|jS) aReturns a tf1.placeholder object given optional hints, such as a space. Note that the returned placeholder will always have a leading batch dimension (None). Args: space: An optional gym.Space to hint the shape and dtype of the placeholder. value: An optional value to hint the shape and dtype of the placeholder. name: An optional name for the placeholder. time_axis: Whether the placeholder should also receive a time dimension (None). flatten: Whether to flatten the given space into a plain Box space and then create the placeholder from the resulting space. Returns: The tf1 placeholder. r) ModelCatalogNclt|dzdd|DzS)N.c,g|]}t|Sstrrips r8rkz5get_placeholder....s3I3I3IqCFF3I3I3Ir:)r`rh)get_placeholderjoin)path componentrhs r8z!get_placeholder.. s@O'!CZ#((3I3ID3I3I3I*J*JJ---r:Nrz)rRdtyperhr?)ray.rllib.models.catalogrvrSgymspacesDictTupleget_action_placeholderrNmap_structure_with_pathrtf1 placeholderrRrnpfloat64r-rXtupleas_list)r`r(rhrIrOrvrRs ` r8rrsx8655555  ecjosz/?@ A A  #::5$GGG3/u55  9WWr:U[H % rz 9 9"**u{       ABB#+ww-"5%00LuueEMMOO6L6LN!& rz 9 9"**u{    r:orig_clsr"ror)r"rr c||}|dd}|dvrtstd|dkrtsttsJddlm}ddlm}dd l m }t|d r%t||s| }n3t||sn"td ||d r&t|||fr|}|S) a[Returns the corresponding tf-eager class for a given TFPolicy class. Args: orig_cls: The original TFPolicy class to get the corresponding tf-eager class for. config: The Algorithm config dict or AlgorithmConfig object. Returns: The tf eager policy class corresponding to the given TFPolicy class. frameworkr-)tf2r-zCould not import tensorflow!rrrrr!as_eagerz0This policy does not support eager execution: {} eager_tracing)getr ImportErrorexecuting_eagerlyenable_eager_execution ray.rllib.policy.eager_tf_policyr#ray.rllib.policy.eager_tf_policy_v2r ray.rllib.policy.tf_policyr"hasattr issubclassr ValueErrorformat with_tracing)rroclsrrr r"s r8get_tf_eager_cls_if_necessaryr#su C ;--IM!!#!8999E$$&& )  & & ( ( ($$&&&&&BBBBBBGGGGGG777777 8Z ( ( Hm1T1T ##%%CCHh//  BII(SS  ::o & & %: -1, ,  %""$$C Jr:?xdeltactt||ktj|dz|t|d|zz zS)aComputes the huber loss for a given term and delta parameter. Reference: https://en.wikipedia.org/wiki/Huber_loss Note that the factor of 0.5 is implicitly included in the calculation. Formula: L = 0.5 * x^2 for small abs x (delta threshold) L = delta * (abs(x) - 0.5*delta) for larger abs x (delta threshold) Args: x: The input term, e.g. a TD error. delta: The delta parmameter in the above formula. Returns: The Huber loss resulting from `x` and `delta`. ?)r-whereabsmathsquare)rrs r8 huber_lossrUs]$ 88 q E qC S5[()  r:cndtt|dzS)zComputes half the L2 norm over a tensor's values without the sqrt. output = 0.5 * sum(x ** 2) Args: x: The input tensor. Returns: 0.5 times the L2 norm over the given tensor's values (w/o sqrt). r@)r- reduce_sumpowrs r8l2_lossrns' rvva~~.. ..r:session_or_nonez tf1.Session dynamic_shapecZtrJnJfd}|S)aLReturns a function that can be executed in either graph or eager mode. The function must take only positional args. If eager is enabled, this will act as just a function. Otherwise, it will build a function that executes a session run with placeholders internally. Args: session_or_none: tf.Session if in graph mode, else None. dynamic_shape: True if the placeholders should have a dynamic batch dimension. Otherwise they will be fixed shape. Returns: A function that can be called in either eager or static-graph mode. Nc:gidgfd}|SS)Nc g}|D]C}t|tur||.||D|}dj5 fd}t j||}t j|D]} |t j||}| D] \}}| |<  i d<dddn #1swxYwYtt t j| t j fd | d }|S)Nrcr.t|jdkrd|jddz}n d}n|j}t|j|dd|DS)Nrrr?rzrxc,g|]}t|Srzr{r}s r8rkzfmake_tf_callable..make_wrapper..call.._create_placeholders..s.D.D.D!s1vv.D.D.Dr:)rrRrh)rPrRrrrr)rr(rRrs r8_create_placeholderszRmake_tf_callable..make_wrapper..call.._create_placeholderss,4#&u{#3#3a#7#7,3ek!""o,EEE,.EE(- #&??&+k&+%(XX.D.Dt.D.D.D%E%E$3$$r:c0||Sr) __setitem__)phr6 feed_dicts r8rzFmake_tf_callable..make_wrapper..call..s)"7"7A">">r:)typer1extendrUgraph as_defaultrNrrOr,dictr3 map_structurerun)argskwargs args_flatar placeholdersrr5retrargs_placeholdersrfnkwargs_placeholdersr symbolic_outs @r8callz4make_tf_callable..make_wrapper..calls ,,AAww$!((++++!((++++   ?*(.99;;XX     (,'C0$(( #',|"<"<99B-44R8888'+'C0&(( &2%7%7%9%988EAr57/22*,".?*WCV*W*W Q9XXXXXXXXXXXXXXX:!%6 T8J8J!K!KLL ">>>>' &)),q/9EE s.BDDDrz)rrrrrrrs` @@@r8 make_wrapperz&make_tf_callable..make_wrappersc  & " "$  6L1 1 1 1 1 1 1 1 1 1 fKIr:)r-r)rrrs`` r8make_tf_callabler}sb* +&&&&***@@@@@@D r:$@ optimizer objectivevar_listz tf.Variableclip_valc 0dks JtrF|j}tt t||||}n|||}fd|DS)aComputes, then clips gradients using objective, optimizer and var list. Ensures the norm of the gradients for each variable is clipped to `clip_val`. Args: optimizer: Either a shim optimizer (tf eager) containing a tf.GradientTape under `self.tape` or a tf1 local optimizer object. objective: The loss tensor to calculate gradients on. var_list: The list of tf.Variables to compute gradients over. clip_val: The global norm clip value. Will clip around -clip_val and +clip_val. Returns: The resulting model gradients (list or tuples of grads + vars) corresponding to the input `var_list`. N)rc\g|](\}}|t|n||f)Sr)r-r/)rigr6rs r8rkz%minimize_and_clip..sG    Q =*2)=H % % %1aH ==r:)r-rtaper1r3gradientcompute_gradients)rrrrrgrads_and_varss ` r8minimize_and_cliprs4  x#~~~x~~ - S~c$t}}Y'I'I"J"JHUUVV"44Y4RR    $   r:c:t|tr,t|jtjSt|t rt|jdtj rBtj |j}t j ddfn|j}t fdt|DdStd|)ajReturns a one-hot tensor, given and int tensor and a space. Handles the MultiDiscrete case as well. Args: x: The input tensor. space: The space to use for generating the one-hot tensor. Returns: The resulting one-hot tensor. Raises: ValueError: If the given space is not a discrete one. .. testcode:: :skipif: True import gymnasium as gym import tensorflow as tf from ray.rllib.utils.tf_utils import one_hot x = tf.Variable([0, 3], dtype=tf.int32) # batch-dim=2 # Discrete space with 4 (one-hot) slots per batch item. s = gym.spaces.Discrete(4) one_hot(x, s) .. testoutput:: .. testcode:: :skipif: True x = tf.Variable([[0, 1, 2, 3]], dtype=tf.int32) # batch-dim=1 # MultiDiscrete space with 5 + 4 + 4 + 7 = 20 (one-hot) slots # per batch item. s = gym.spaces.MultiDiscrete([5, 4, 4, 7]) one_hot(x, s) .. testoutput:: rrrKcxg|]6\}}tdd|f|tj7S)Nr)r-rWrX)riinrs r8rkzone_hot..<s= R R R$!QRZZ!!!Q$"*Z 5 5 R R Rr:rLz#Unsupported space for `one_hot`: {})rSr r-rWrrXr nvecrndarrayravelrTrRrY enumeraterr)rr`rs` r8rWrWsX%"" Nzz!UWBJz777 E= ) ) N ejmRZ 0 0 8EJ''D 1qwqz2.//AA:Dyy R R R R)D// R R R   >EEeLLMMMr:rMct|tjj}t||t|}tj||tjt|tj|z S)zSame as tf.reduce_mean() but ignores -inf values. Args: x: The input tensor to reduce mean over. axis: The axis over which to reduce. None for all axes. Returns: The mean reduced inputs, ignoring inf values. ) r- not_equalrXminr zeros_likerrrV)rrMmaskx_zeroeds r8reduce_mean_ignore_infrCs <<2:> * *Dxxaq!1!122H 7  h - -0B0B bj!!411 r:scopeztf1.VariableScopetrainable_onlyct|rtjjntjjt |t r|n|jS)aGet variables inside a given scope. Args: scope: Scope in which the variables reside. trainable_only: Whether or not to return only the variables that were marked as trainable. Returns: The list of variables in the given `scope`. )r)rget_collection GraphKeysTRAINABLE_VARIABLES VARIABLESrSr|rh)rrs r8 scope_varsrUsS     % )) ] $!%--=ee5:   r: tf.Tensorctj|tjtj|dzzS)zThe symlog function as described in [1]: [1] Mastering Diverse Domains through World Models - 2023 D. Hafner, J. Pasukonis, J. Ba, T. Lillicrap https://arxiv.org/pdf/2301.04104v1.pdf r?)r-rsignlogrrs r8symlogrks: 7<<??RW[[Q!);<< <>r:44@ num_buckets lower_bound upper_boundcdt|||}ttdt|d|p tj}||z |dz z }| |z|z }tj|}tj|} t t || | dz| } t t | || dz | } |||zz} || |zz} || z | | z z } d| z } t ||gd}t || gd}t ||gd}t | | gd}t t|tj|t|d|fS)a+Returns a two-hot vector of dim=num_buckets with two entries that are non-zero. See [1] for more details: [1] Mastering Diverse Domains through World Models - 2023 D. Hafner, J. Pasukonis, J. Ba, T. Lillicrap https://arxiv.org/pdf/2301.04104v1.pdf Entries in the vector represent equally sized buckets within some fixed range (`lower_bound` to `upper_bound`). Those entries not 0.0 at positions k and k+1 encode the actual `value` and sum up to 1.0. They are the weights multiplied by the buckets values at k and k+1 for retrieving `value`. Example: num_buckets=11 lower_bound=-5 upper_bound=5 value=2.5 -> [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.5, 0.0, 0.0] -> [-5 -4 -3 -2 -1 0 1 2 3 4 5] (0.5*2 + 0.5*3=2.5) Example: num_buckets=5 lower_bound=-1 upper_bound=1 value=0.1 -> [0.0, 0.0, 0.8, 0.2, 0.0] -> [-1 -0.5 0 0.5 1] (0.2*0.5 + 0.8*0=0.1) Args: value: The input tensor of shape (B,) to be two-hot encoded. num_buckets: The number of buckets to two-hot encode into. lower_bound: The lower bound value used for the encoding. If input values are lower than this boundary, they will be encoded as `lower_bound`. upper_bound: The upper bound value used for the encoding. If input values are higher than this boundary, they will be encoded as `upper_bound`. Returns: The two-hot encoded tensor of shape (B, num_buckets). rrr?rrrK)rR)r-r.rVrangerRrXrfloorceilrequalstackrY scatter_ndint32)r(rr r r batch_indices bucket_deltaidxr5kp1values_k values_kp1 weights_k weights_kp1 indices_k indices_kp1indicesupdatess r8two_hotrsb   UK = =EGG BHHUOOA&''!rzM  +-+/BL <% < /C  cA ',,s  C ((288As##S3Y 4 4C ((288C--sSy# > >CQ--Hs\11J#:(=>I /K-+R00I((M3/44KiiK0!44GiiK0!44G == ""xxq!;/   r:main_net target_nettauct|j|jD](\}}||zd|z |zz}||)dS)a`Updates a keras.Model target network using Polyak averaging. new_target_net_weight = ( tau * main_net_weight + (1.0 - tau) * current_target_net_weight ) Args: main_net: The keras.Model to update from. target_net: The target network to update. tau: The tau value to use in the Polyak averaging formula. rN)r3 variablesassign)r r!r"old_var current_var updated_vars r8update_target_networkr)s_"!$J$8(:L M M$$K'39*?? {####$$r:actionsctj|d}t|tj}t |jdkr$|dddf}t |jdk$|S)aHelper function useful for returning dummy logp's (0) for some actions. Args: actions: The input actions. This can be any struct of complex action components or a simple tensor of different dimensions, e.g. [B], [B, 2], or {"a": [B, 4, 5], "b": [B]}. Returns: A 1D tensor of 0.0 (dummy logp's) matching the batch dim of `actions` (shape=[B]). rrr?N)rNrOr-rrXrPrR)r*action_componentlogp_s r8zero_logps_from_actionsr.su|G,,Q/ MM*"*M = =E ek  Q  aaad  ek  Q   Lr:policymean_klcd|r@ttj|dfddSdS)NcjtdtdS)Na}KL divergence is non-finite, this will likely destabilize your model and the training process. Action(s) in a specific state have near-zero probability. This can happen naturally in deterministic environments where the optimal policy has zero mass for a specific action. To fix this issue, consider setting the coefficient for the KL loss term to zero or increasing policy entropy.r)loggerwarningr-constantrzr:r8 print_warningz5warn_if_infinite_kl_divergence..print_warnings1 *   {{3r:c6tdS)Nr)r-r5rzr:r8rz0warn_if_infinite_kl_divergence..,sR[[--r:cSrrz)r6sr8rz0warn_if_infinite_kl_divergence..-s MMOOr:)false_fntrue_fn)loss_initializedr-condris_inf)r/r0r6s @r8warn_if_infinite_kl_divergencer>sz         GNN7 # #--++++        r:cd}g}|D]\}tj|tj}||||z|}||||z }]t ||ks Jd|S)Nrrz.Passed weight does not have the correct shape.)rprodint_rTrUrP)vectorshapesrarraysrRsizearrays r8 _unflattenrG1s A FwuBG,,,qAH~&..u55 e T  v;;!   M    Mr:cDeZdZdZd dZdZdZdZdZde fd Z d Z dS) TensorFlowVariablesaA class used to set and get weights for Tensorflow networks. Attributes: sess (tf.Session): The tensorflow session used to run assignment. variables (Dict[str, tf.Variable]): Extracted variables from the loss or additional variables that are passed in. placeholders (Dict[str, tf.placeholders]): Placeholders for weights. assignment_nodes (Dict[str, tf.Tensor]): Nodes that assign weights. Nc ||_t|}t|}g t|}t |dkr|}|*t |dr|j}|jD]0}||vr*| || |1|j D]0}||vr*| || |1d|j jvsd|j jvr |j j t |dkt|_ fdt D} || |z } t s| D]} | |j| jj j <i|_i|_|jD]\} } t | j| d| z|j| <| |j| |j| <dS| D]} | |j| j <dS) aUCreates TensorFlowVariables containing extracted variables. The variables are extracted by performing a BFS search on the dependency graph with loss as the root node. After the tree is traversed and those variables are collected, we append input_variables to the collected variables. For each variable in the list, the variable has a placeholder and assignment operation created for it. Args: output (tf.Operation, List[tf.Operation]): The tensorflow operation to extract all variables from. sess (Optional[tf.Session]): Optional tf.Session used for running the get and set methods in tf graph mode. Use None for tf eager. input_variables (List[tf.Variables]): Variables to include in the list. rNopVariable VarHandlec:g|]}|jjjv|Srz)rKnode_defrh)rir6variable_namess r8rkz0TensorFlowVariables.__init__..ys1   1C~1U1UA1U1U1Ur: Placeholder_)rh)sessrrsetrPpopleftrrKrGrUaddcontrol_inputsrOrhrr$rglobal_variablesrrassignment_nodesr,rr(r get_shaperr%)selfoutputrRinput_variablesqueueexplored_inputstf_objinput_opcontrol variable_listr6r5varrPs @r8__init__zTensorFlowVariables.__init__Is$ F##f f++%jjAoo]]__F~vt$$ #"M 2 2?22LL***#''111"0 1 1/11LL)))#''000V_///;&/BT3T3T%%fo&:;;;)%jjAoo*%    ++--     & _ ,M$$&& +" 7 756qt}122 "D $&D !...00 L L3'*IIKK%MMOO++--'!+(7((!!$ ,/::d6G6J+K+K%a((  L L# + +)*qv&& + +r:cbtd|jDS)zReturns the total length of all of the flattened variables. Returns: The length of all flattened variables concatenated. c3K|]<}tj|V=dSr)rr@rYrrir6s r8 z4TensorFlowVariables.get_flat_size..s@UU271;;==002233UUUUUUr:)sumr$r2rZs r8 get_flat_sizez!TensorFlowVariables.get_flat_sizes/ UUT^=R=R=T=TUUUUUUr:cjs5tjdjDStjfdjDS)zGets the weights and returns them as a flat array. Returns: 1D Array containing the flattened weights. cZg|](}|)Srz)numpyrOrgs r8rkz0TensorFlowVariables.get_flat..s,FFF""$$FFFr:cjg|]/}|j0S))session)evalrRrOrir6rZs r8rkz0TensorFlowVariables.get_flat..s5 R R RQQVVDIV & & . . 0 0 R R Rr:)rRr concatenater$r2rjs`r8get_flatzTensorFlowVariables.get_flats}y >FFdn.C.C.E.EFFF ~ R R R R$.:O:O:Q:Q R R R   r:c djD}t||}jsDt j|D]\}}||dSfdjD}jtj tt ||dS)a;Sets the weights to new_weights, converting from a flat array. Note: You can only set all weights in the network using this function, i.e., the length of the array must match get_flat_size. Args: new_weights (np.ndarray): Flat array containing weights. cZg|](}|)Srz)rYrrgs r8rkz0TensorFlowVariables.set_flat..s,KKKa!++--''))KKKr:c0g|]\}}j|Srz)r)rir5r6rZs r8rkz0TensorFlowVariables.set_flat..s%TTTTQD-a0TTTr:rN) r$r2rGrRr3r%r,rr1rXr)rZ new_weightsrCrDr6rrs` r8set_flatzTensorFlowVariables.set_flatsLK4>3H3H3J3JKKKK00y DN1133V<<  1   UTTTT^=Q=Q=S=STTTL IMMT*113344s<8899      r:c\|js|jS|j|jS)zReturns a dictionary containing the weights of the network. Returns: Dictionary mapping variable names to their weights. )rRr$rrjs r8 get_weightszTensorFlowVariables.get_weightss,y "> !y}}T^,,,r:ryc|j<|jD] \}}|||!dS||\}}|j||dS)aSets the weights to new_weights. Note: Can set subsets of variables as well, by only passing in the variables you want to be set. Args: new_weights: Dictionary mapping variable names to their weights. Nrx)rRr$r,r%_assign_weightsr)rZryrhrc assign_listrs r8 set_weightszTensorFlowVariables.set_weightss 9 !^1133 . . c ;t,---- . .&*%9%9+%F%F "K IMM+M ; ; ; ; ;r:c gitjd fd}|D]j\ vr |  fdD} fdt |dD}|r||d ks Jdt t |ks Jd fd DfS) a'Sets weigths using exact or closest assignable variable name Args: weights: Dictionary mapping variable names to their weights. Returns: Tuple[List, Dict]: assigned variables list, dict of placeholders and weights c:tfd|DS)Ncg|]}|v| Srzrz)riel2s r8rkzOTensorFlowVariables._assign_weights..nb_common_elem..s111abr:)rP)l1rs `r8nb_common_elemz;TensorFlowVariables._assign_weights..nb_common_elems&1111211122 2r:c||j|<||dSr)rrUremove)rhr( assignableassignedrrZs r8r%z3TensorFlowVariables._assign_weights..assignsA16Id'- . OOD ! ! !   d # # # # #r:c vi|]5}|d|d6S)/)split)rircrhrs r8 z7TensorFlowVariables._assign_weights..sK 33HHr:c\g|](\}}|dk jj|jk&|)S)r)rRrX)ri close_varcnrZr(s r8rkz7TensorFlowVariables._assign_weights..sG% 2Avv%+1Fy1Q1W"W"W"W"W"Wr:c|d S)Nr?rz)rs r8rz5TensorFlowVariables._assign_weights..s qQRter:)keyrzNo variables in the input matched those in the network. Possible cause: Two networks were defined in the same TensorFlow graph. To fix this, place each network definition in its own tf.Graph.z^All weights couldn't be assigned because no variable had an exact/close name or had same shapec*g|]}j|Srz)rXrrs r8rkz7TensorFlowVariables._assign_weights..s!;;;Q%a(;;;r:)rSrXr4r,sortedrP) rZweightsr%commonselectrrrrhrr(s ` @@@@@@r8r~z#TensorFlowVariables._assign_weightss .335566  3 3 3 $ $ $ $ $ $ $ $ #==?? - -KD%z!!tU####))/ OO)T)T)T -F6!9e,,,   .  x8}}G ,,, 8-,, <;;;(;;;YFFr:)NN) __name__ __module__ __qualname____doc__rdrkrtrzr|rrr~rzr:r8rIrI=sG+G+G+G+RVVV   , - - -rGrIrzr:r8rsj********LLLLLLLLLLLLLLLLLL 44444444&&&&&&????????333333......IIIIII4EEEEEE888888>>>>>>CCCCCC333333  8 $ $} R "&----- - e_ --- -` E*EJE:EEE E$ ,0ee eK(ee eee eP ?c??? ?, "& 7 7 7 CI 7  C=7  3- 7  7  7 7 7 7  7 t .:. #%?? @. %> ?@... .b *UZ 0  /z /j / / /  / DI[[m,[=A[ [[[ [@  %%%%=!% %  %%% %P 8Nz8N#)8N 8N8N8N 8Nv j  " CH )) *<@ - * === = ??? ?,   YY YYY YYY Yx $$$ $ $$$ $* %5* 2   '1      ,   SGSGSGSGSGSGSGSGSGSGr: