mstar.utils.fp8_block

mstar.utils.fp8_block#

FP8 block-scale checkpoints: the quantization_config they declare, and their weights dequantized.

A block-scale checkpoint (DeepSeek-V3’s format, GLM-5.x’s) stores each linear weight as e4m3 with one fp32 weight_scale_inv per weight_block_size block. Triton-free, so a model’s config can import it on a CPU-only install; the activation quantizer that feeds the block-scale GEMMs is quant_fp8.

Functions

dequant_fp8_block_stream(weights, quant_config)

Pair fp8 .weight keys with .weight_scale_inv and dequantize.

dequantize_fp8_block_weight(weight_fp8, ...)

Dequantize one 2-D fp8 weight: w[i, j] * scale_inv[i//bo, j//bi].

Classes

Fp8BlockQuantConfig([weight_block_size, ...])

The fields of an HF fp8 quantization_config a block-scale checkpoint carries.
class mstar.utils.fp8_block.Fp8BlockQuantConfig(weight_block_size=(128, 128), fmt='e4m3', activation_scheme='dynamic', ignore=<factory>)[source]#

Bases: object

The fields of an HF fp8 quantization_config a block-scale checkpoint carries.

Parameters:
activation_scheme: str = 'dynamic'#
fmt: str = 'e4m3'#
classmethod from_hf_config_dict(quant)[source]#
Parameters:

quant (dict | None)

Return type:

Fp8BlockQuantConfig | None

ignore: tuple[str, ...]#
weight_block_size: tuple[int, int] = (128, 128)#
mstar.utils.fp8_block.dequant_fp8_block_stream(weights, quant_config, out_dtype=torch.bfloat16, keep_fp8=None)[source]#

Pair fp8 .weight keys with .weight_scale_inv and dequantize.

Parameters:
Return type:

Iterator[tuple[str, Tensor]]

mstar.utils.fp8_block.dequantize_fp8_block_weight(weight_fp8, scale_inv, block_size=(128, 128), out_dtype=torch.bfloat16)[source]#

Dequantize one 2-D fp8 weight: w[i, j] * scale_inv[i//bo, j//bi].

Parameters:
Return type:

Tensor