Pi( &ddlmZddlmZddlmZmZddlmZeeeeeeeeeeed Z dee effd Z d S) )Dict) Replicate)ColwiseParallelRowwiseParallel) ParallelStyle) input_layouts)output_layouts) tok_embeddingsoutputzlayers.*.attn.q_projzlayers.*.attn.k_projzlayers.*.attn.v_projzlayers.*.attn.output_projzlayers.*.mlp.w1zlayers.*.mlp.w2zlayers.*.mlp.w3returnctS)z Helper function to get the base tensor parallel plan for Llama3 model, which will also be shared with 3.1, 3.2, and 3.3 models Returns: Dict[str, Any]: The tensor parallel plan for Llama3 model. )BASE_LLAMA_TP_PLANx/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchtune/models/llama3/_parallelism.pybase_llama_tp_planrs  rN) typingrtorch.distributed._tensorr!torch.distributed.tensor.parallelrr'torch.distributed.tensor.parallel.stylerrstrrrrrrs //////NNNNNNNNAAAAAA &oIIKK@@@oYY[[999+O--+O--+O--!0!2!2&((&((&((  Dm!34r