mstar.model.voxcpm2.config

Contents

mstar.model.voxcpm2.config#

Checkpoint-backed configuration for VoxCPM2 (openbmb/VoxCPM2).

config.json describes every component: the MiniCPM-4 text-semantic LM, the residual acoustic LM, the local encoder and local DiT (both MiniCPM-4 stacks read bidirectionally), and the AudioVAE V2 (16 kHz in, 48 kHz out). The dataclasses below mirror that file; anything the checkpoint does not carry is a generation default of the reference VoxCPM.generate.

Classes

AudioVAEConfig([encoder_dim, encoder_rates, ...])

MiniCPM4Config([hidden_size, ...])

One MiniCPM-4 stack; the residual LM, encoder and DiT override sizes.

VoxCPM2Config(lm, residual, encoder, dit, ...)

class mstar.model.voxcpm2.config.AudioVAEConfig(encoder_dim: 'int' = 128, encoder_rates: 'tuple[int, ...]' = (2, 5, 8, 8), latent_dim: 'int' = 64, decoder_dim: 'int' = 2048, decoder_rates: 'tuple[int, ...]' = (8, 6, 5, 2, 2, 2), depthwise: 'bool' = True, sample_rate: 'int' = 16000, out_sample_rate: 'int' = 48000, use_noise_block: 'bool' = False, sr_bin_boundaries: 'tuple[int, ...] | None' = (20000, 30000, 40000), cond_type: 'str' = 'scale_bias', cond_dim: 'int' = 128, cond_out_layer: 'bool' = False)[source]#

Bases: object

Parameters:

  • encoder_dim (int)

  • encoder_rates (tuple[int, ...])

  • latent_dim (int)

  • decoder_dim (int)

  • decoder_rates (tuple[int, ...])

  • depthwise (bool)

  • sample_rate (int)

  • out_sample_rate (int)

  • use_noise_block (bool)

  • sr_bin_boundaries (tuple[int, ...] | None)

  • cond_type (str)

  • cond_dim (int)

  • cond_out_layer (bool)

cond_dim: int = 128#
cond_out_layer: bool = False#
cond_type: str = 'scale_bias'#
property decode_hop: int#

25 Hz).

Type:

48 kHz output samples per latent frame (1920

decoder_dim: int = 2048#
decoder_rates: tuple[int, ...] = (8, 6, 5, 2, 2, 2)#
depthwise: bool = True#
property encode_hop: int#

25 Hz).

Type:

16 kHz input samples per latent frame (640

encoder_dim: int = 128#
encoder_rates: tuple[int, ...] = (2, 5, 8, 8)#
latent_dim: int = 64#
out_sample_rate: int = 48000#
sample_rate: int = 16000#
sr_bin_boundaries: tuple[int, ...] | None = (20000, 30000, 40000)#
use_noise_block: bool = False#
class mstar.model.voxcpm2.config.MiniCPM4Config(hidden_size=2048, intermediate_size=6144, num_hidden_layers=28, num_attention_heads=16, num_key_value_heads=2, head_dim=128, rms_norm_eps=1e-05, vocab_size=73448, rope_theta=10000.0, max_position_embeddings=32768, rope_short_factor=(), rope_long_factor=(), rope_original_max_position_embeddings=32768, no_rope=False, use_mup=False)[source]#

Bases: object

One MiniCPM-4 stack; the residual LM, encoder and DiT override sizes.

Parameters:

  • hidden_size (int)

  • intermediate_size (int)

  • num_hidden_layers (int)

  • num_attention_heads (int)

  • num_key_value_heads (int)

  • head_dim (int)

  • rms_norm_eps (float)

  • vocab_size (int)

  • rope_theta (float)

  • max_position_embeddings (int)

  • rope_short_factor (tuple[float, ...])

  • rope_long_factor (tuple[float, ...])

  • rope_original_max_position_embeddings (int)

  • no_rope (bool)

  • use_mup (bool)

head_dim: int = 128#
hidden_size: int = 2048#
intermediate_size: int = 6144#
max_position_embeddings: int = 32768#
no_rope: bool = False#
num_attention_heads: int = 16#
num_hidden_layers: int = 28#
num_key_value_heads: int = 2#
resized(*, hidden_size, intermediate_size, num_attention_heads, num_hidden_layers, head_dim)[source]#

A stack sharing this one’s RoPE, norms and KV head count (the reference deep-copies lm_config and overrides the sizes).

Parameters:

  • hidden_size (int)

  • intermediate_size (int)

  • num_attention_heads (int)

  • num_hidden_layers (int)

  • head_dim (int | None)

Return type:

MiniCPM4Config

rms_norm_eps: float = 1e-05#
property rope_attention_scale: float#
property rope_factors: tuple[float, ...]#

LongRoPE per-dimension divisors, picked as MiniCPMLongRoPE does for its cache length (max_position_embeddings).

rope_long_factor: tuple[float, ...] = ()#
rope_original_max_position_embeddings: int = 32768#
rope_short_factor: tuple[float, ...] = ()#
rope_theta: float = 10000.0#
use_mup: bool = False#
vocab_size: int = 73448#
class mstar.model.voxcpm2.config.VoxCPM2Config(lm: 'MiniCPM4Config' = <factory>, residual: 'MiniCPM4Config' = <factory>, encoder: 'MiniCPM4Config' = <factory>, dit: 'MiniCPM4Config' = <factory>, audio_vae: 'AudioVAEConfig' = <factory>, patch_size: 'int' = 4, feat_dim: 'int' = 64, fsq_latent_dim: 'int' = 512, fsq_scale: 'int' = 9, dit_mean_mode: 'bool' = False, max_length: 'int' = 8192, cfg_value: 'float' = 2.0, inference_timesteps: 'int' = 10, min_len: 'int' = 2, max_len: 'int' = 4096, max_len_ratio: 'float' = 6.0, prompt_context_patches: 'int' = 3)[source]#

Bases: object

Parameters:
audio_vae: AudioVAEConfig#
cfg_value: float = 2.0#
dit: MiniCPM4Config#
dit_mean_mode: bool = False#
encoder: MiniCPM4Config#
feat_dim: int = 64#
classmethod from_dict(data)[source]#
Parameters:

data (dict[str, Any])

Return type:

VoxCPM2Config

classmethod from_pretrained(path)[source]#
Parameters:

path (str | Path)

Return type:

VoxCPM2Config

fsq_latent_dim: int = 512#
fsq_scale: int = 9#
inference_timesteps: int = 10#
lm: MiniCPM4Config#
max_len: int = 4096#
max_len_ratio: float = 6.0#
max_length: int = 8192#
max_patches(target_text_tokens, max_len=None)[source]#
Parameters:

  • target_text_tokens (int)

  • max_len (int | None)

Return type:

int

min_len: int = 2#
property output_sample_rate: int#
property patch_samples: int#

7680, 160 ms.

Type:

48 kHz samples one LM step (one patch) decodes to

patch_size: int = 4#
prompt_context_patches: int = 3#
property ref_patch_samples: int#
Type:

16 kHz samples of reference audio per patch

residual: MiniCPM4Config#
Contents