)`i dZddlZddlmZddlmZddlZddlmZddl m Z ddl m Z m Z mZmZdd lmZejd efd Zd ejd ejddfdZe dd ejdejdeedejfdZe dd ejdejdeedejfdZe dd ejdejdeedejfdZedZdS)a3 Copyright (c) 2024 by FlashInfer team. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. N)SimpleNamespace)Optional)flashinfer_api)gen_act_and_mul_module)device_support_pdlregister_custom_opregister_fake_opget_compute_capability)get_fp4_quantization_module act_func_namec t|}|d}t||td|d d dtjdtjdt tddffd }td| d dtjdtjdt tddfd }td i||iS) N_and_mulz flashinfer::)out) mutates_argsrinput enable_pdlreturncN|t|j}|||dSN)rdevice)rrrfns i/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/flashinfer/activation.py _act_and_mulz,get_act_and_mul_module.._act_and_mul*s4  +EL99J 3z"""""cdSr)rrrs r_fake_act_and_mulz1get_act_and_mul_module.._fake_act_and_mul2s  rrr) rbuild_and_loadgetattrr torchTensorrboolr r)r modulefnamerrrs @rget_act_and_mul_moduler&"s( #M 2 2 A A C CF & & &E   B.u..XFFFMQ## \#"',#`_ Returns ------- output: torch.Tensor Output tensor, shape (..., hidden_size). Nr)r*The pointers must be multiple of 16 bytes.r*rdtypesilu) rrr,r2itemsize ValueErrorr-r!emptyr& silu_and_mulrrrs rr7r7Fs2' 55  {2--2a77EFFF UC    k K  B1 46 6<+    6""//   Jrc|t|j}|jd|jjzdzdkrt d|t ||n@tj|jdd|jddzfz|j|j}td ||||S) aFused GeLU Tanh and Mul operation. ``gelu(tanh(input[..., :hidden_size])) * input[..., hidden_size:]`` Parameters ---------- input: torch.Tensor Input tensor, shape (..., 2 * hidden_size). out: Optional[torch.Tensor] The output tensor, if specified, the kernel will update this tensor inplace. enable_pdl: bool Whether to enable `programmatic dependent launch `_ Returns ------- output: torch.Tensor Output tensor, shape (..., hidden_size). Nr)r/rr0r*r1 gelu_tanh) rrr,r2r4r5r-r!r6r&gelu_tanh_and_mulr8s rr;r;ss2' 55  {2--2a77EFFF UC    k K  B1 46 6<+    ;''99#ujQQQ Jrc|t|j}|jd|jjzdzdkrt d|t ||n@tj|jdd|jddzfz|j|j}td ||||S) awFused GeLU and Mul operation. ``gelu(input[..., :hidden_size]) * input[..., hidden_size:]`` Parameters ---------- input: torch.Tensor Input tensor, shape (..., 2 * hidden_size). out: Optional[torch.Tensor] The output tensor, if specified, the kernel will update this tensor inplace. enable_pdl: bool Whether to enable `programmatic dependent launch `_ Returns ------- output: torch.Tensor Output tensor, shape (..., hidden_size). Nr)r/rr0r*r1gelu) rrr,r2r4r5r-r!r6r& gelu_and_mulr8s rr>r>s2' 55  {2--2a77EFFF UC    k K  B1 46 6<+    6""//UJGGG Jrct|j\}}|dz|z}t||||\}}||fS)a Silu and multiply and quantize batched input tensor to NVFP4 format with mask. Parameters: a (torch.Tensor): Input tensor of shape [B, M, K] with dtype fp16/bf16. a_global_sf (torch.Tensor): Global scale factor of shape [1] with dtype float32. mask (torch.Tensor): Mask tensor to apply before quantization. sf_vec_size (int, optional): Scale factor vector size. Defaults to 16. Returns: Tuple[torch.Tensor, torch.Tensor]: A tuple containing: - Quantized tensor of shape [B, M, K/2] with dtype FLOAT4_E2M1X2 - Scale factors tensor with shape determined by layout and sf_vec_size )r rr 0silu_and_mul_scaled_nvfp4_experts_quantize_sm100)amask a_global_sfmajorminor device_archa_fp4a_sfs r*silu_and_mul_scaled_nvfp4_experts_quantizerJsf$*!(33LE5RZ%')K-66  E4 $;r)NN)__doc__ functoolstypesrtypingrr! api_loggingrjitrutilsrr r r fp4_quantizationr cachestrr&r"r-r#r7r;r>rJrrrrUs2 !!!!!! '''''''''''' :99999 4#44442 eltPT)) <)#l)?G~) \))))XPT%% <%#l%?G~% \%%%%PPT%% <%#l%?G~% \%%%%Pr