Pi ddlZddlmZddlmZmZddlZddlmZddl m Z ddl m Z ddl mZmZmZddlmZmZdd lmZd ZeGd d e ZeZeed ejjdefdZGdde ZeZeed ejjdedejjfdZ ddejjde deeejjege fdejjfdZ!dS)N) dataclass)CallableOptional)to_sparse_semi_structured) AOBaseConfig)WeightNormSparsifier) _is_linear_linear_extra_repr)_replace_with_custom_fn_if_matches_filter)_QUANTIZE_CONFIG_HANDLER register_quantize_module_handler)BlockSparseTensorc `|dt}g}|D]+\}}|||r|d|di,t ddd}|||||dS) zO This function simulates 2:4 sparsity on all linear layers in a model. filter_fn tensor_fqnz.weightg?))sparsity_levelsparse_block_shapezeros_per_blockN)popr named_modulesappendrpreparestep squash_mask)modelkwargsr sparse_confignamemod sparsifiers o/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchao/sparsity/sparse_api.pyapply_fake_sparsityr%s ; 33IM((**CC c 9S$   C  ,40@0@0@!A B B B%vqJum,,,OOc$eZdZUdZeed<dZdS)BlockSparseWeightConfig@ blocksizecDtjddS)Nz(torchao.sparsity.BlockSparseWeightConfigtorch_C_log_api_usage_onceselfs r$ __post_init__z%BlockSparseWeightConfig.__post_init__5s $$%OPPPPPr&N)__name__ __module__ __qualname__r*int__annotations__r2r&r$r(r(1s:IsQQQQQr&r(moduleconfigc|j}tj|j|}tj|d|_tjt||_ |SNF) requires_grad) r*r from_denseweightr-nn Parametertypes MethodTyper extra_repr)r9r:r* new_weights r$_block_sparse_weight_transformrF=sW  I"-fmYGGJH&&z&GGFM();VDDF Mr&ceZdZdZdZdS)SemiSparseWeightConfigze Configuration for converting the weight of linear modules to semi-structured (2:4) sparsity cDtjddS)Nz'torchao.sparsity.SemiSparseWeightConfigr,r0s r$r2z$SemiSparseWeightConfig.__post_init__Ns $$%NOOOOOr&N)r3r4r5__doc__r2r8r&r$rHrHIs2PPPPPr&rHreturnct|j}tj|d|_t jt||_|Sr<) rr?r-r@rArBrCr rD)r9r:rEs r$_semi_sparse_weight_transformrMVsJ +6=99JH&&z&GGFM();VDDF Mr&rrctjdtt |}t |||t n||fdS)aConvert the weight of linear modules in the model with `apply_tensor_subclass`. This function is essentially the same as quantize, put for sparsity subclasses. Currently, we support three options for sparsity: - semi-structured (2:4) sparsity with `semi_sparse_weight` - int8 dynamic quantization + 2:4 sparsity with `layout=SemiSparseLayout` - int4 weight-only quantization + 2:4 sparsity with `layout=SparseMarlinLayout` Args: model (torch.nn.Module): input model config (AOBaseConfig): a workflow configuration object filter_fn (Optional[Callable[[torch.nn.Module, str], bool]]): function that takes a nn.Module instance and fully qualified name of the module, returns True if we want to apply the specified workflow to this module. **Example:** :: import torch import torch.nn as nn from torchao.sparsity import sparsify_ def filter_fn(module: nn.Module, fqn: str) -> bool: return isinstance(module, nn.Linear) m = nn.Sequential(nn.Linear(32, 1024), nn.Linear(1024, 32)) # for 2:4 sparsity from torchao.sparse_api import semi_sparse_weight m = sparsify_(m, semi_sparse_weight(), filter_fn) # for int8 dynamic quantization + 2:4 sparsity from torchao.dtypes import SemiSparseLayout m = quantize_(m, Int8DynamicActivationInt8WeightConfig(layout=SemiSparseLayout), filter_fn) ztorchao.sparsity.sparsify_N) extra_args)r-r.r/r typer r )rr:rhandlers r$ sparsify_rRasdJ H  !=>>>&tF||4G- ' Y9 r&)N)"rB dataclassesrtypingrrr- torch.sparsertorchao.core.configrr`s  !!!!!!%%%%%%%% 222222,,,,,, ;:::::& QQQQQlQQ Q."!"9:: HO #;:PPPPP\PPP,"!"899 HO " X_:9CG,, 8?, ,%(/3!7!=>?, X_ ,,,,,,r&