coreai_opt.quantization.spec.QuantizationGranularity

class coreai_opt.quantization.spec.QuantizationGranularity[source]

Bases: BaseModel, ConfigRegistryMixin

Base class for quantization granularity specifications.

get_block_size(tensor_shape, quantization_target=CompressionTargetTensor.WEIGHT)[source]

Get a list of block sizes based on the granularity.

Parameters:

  • tensor_shape (Size) – Shape of the tensor being quantized.

  • quantization_target (CompressionTargetTensor) – Whether the tensor is a weight or an activation. Defaults to WEIGHT, which preserves the historical behavior.

Return type:

tuple[int, …]