mstar.model.whisper.config

Contents

mstar.model.whisper.config#

Config for Whisper encoder-decoder ASR models (large-v3, large-v3-turbo, …).

Values are read from the HF checkpoint’s config.json, generation_config.json and preprocessor_config.json so one class serves every Whisper size: turbo differs from large-v3 only in decoder_layers (4 vs 32) and its alignment heads.

Classes

WhisperModelConfig(d_model, decoder_layers, ...)

class mstar.model.whisper.config.WhisperModelConfig(d_model: 'int' = 1280, decoder_layers: 'int' = 32, decoder_attention_heads: 'int' = 20, decoder_ffn_dim: 'int' = 5120, encoder_layers: 'int' = 32, encoder_attention_heads: 'int' = 20, encoder_ffn_dim: 'int' = 5120, num_mel_bins: 'int' = 128, vocab_size: 'int' = 51866, max_target_positions: 'int' = 448, max_source_positions: 'int' = 1500, activation_function: 'str' = 'gelu', scale_embedding: 'bool' = False, decoder_start_token_id: 'int' = 50258, eos_token_id: 'int' = 50257, no_timestamps_token_id: 'int' = 50364, prev_sot_token_id: 'int' = 50362, max_initial_timestamp_index: 'int' = 50, lang_to_id: 'dict[str, int]'=<factory>, task_to_id: 'dict[str, int]'=<factory>, suppress_tokens: 'list[int]' = <factory>, begin_suppress_tokens: 'list[int]' = <factory>, alignment_heads: 'list[list[int]]' = <factory>, sampling_rate: 'int' = 16000, hop_length: 'int' = 160, n_fft: 'int' = 400, chunk_length: 'int' = 30)[source]#

Bases: object

Parameters:

  • d_model (int)

  • decoder_layers (int)

  • decoder_attention_heads (int)

  • decoder_ffn_dim (int)

  • encoder_layers (int)

  • encoder_attention_heads (int)

  • encoder_ffn_dim (int)

  • num_mel_bins (int)

  • vocab_size (int)

  • max_target_positions (int)

  • max_source_positions (int)

  • activation_function (str)

  • scale_embedding (bool)

  • decoder_start_token_id (int)

  • eos_token_id (int)

  • no_timestamps_token_id (int)

  • prev_sot_token_id (int)

  • max_initial_timestamp_index (int)

  • lang_to_id (dict[str, int])

  • task_to_id (dict[str, int])

  • suppress_tokens (list[int])

  • begin_suppress_tokens (list[int])

  • alignment_heads (list[list[int]])

  • sampling_rate (int)

  • hop_length (int)

  • n_fft (int)

  • chunk_length (int)

activation_function: str = 'gelu'#
alignment_heads: list[list[int]]#
begin_suppress_tokens: list[int]#
chunk_length: int = 30#
d_model: int = 1280#
decoder_attention_heads: int = 20#
decoder_ffn_dim: int = 5120#
decoder_layers: int = 32#
decoder_prompt_ids(language='en', task='transcribe', timestamps=False, prev_tokens=None)[source]#

The forced decoder prompt.

[<|startofprev|> prev...] <|startoftranscript|> <|lang|> <|task|> [<|notimestamps|>]. With language=None the prompt stops after <|startoftranscript|>: the language token is sampled next (detect_language walk) and prompt_tail_ids() supplies the rest. prev_tokens is the previous window’s transcript for long-form carry-over, kept to the last max_prev_tokens.

Parameters:
Return type:

list[int]

decoder_start_token_id: int = 50258#
encoder_attention_heads: int = 20#
encoder_ffn_dim: int = 5120#
property encoder_head_dim: int#
encoder_layers: int = 32#
eos_token_id: int = 50257#
classmethod from_pretrained(local_dir)[source]#
Parameters:

local_dir (str | Path)

Return type:

WhisperModelConfig

property head_dim: int#
hop_length: int = 160#
is_timestamp(token_id)[source]#
Parameters:

token_id (int)

Return type:

bool

lang_to_id: dict[str, int]#
language_of(token_id)[source]#
Parameters:

token_id (int)

Return type:

str | None

language_token(language)[source]#
Parameters:

language (str)

Return type:

int

property language_token_ids: list[int]#
max_initial_timestamp_index: int = 50#
property max_prev_tokens: int#

How much <|startofprev|> context fits before the prompt proper (openai-whisper / HF: half the context window, minus the marker).

max_source_positions: int = 1500#
max_target_positions: int = 448#
n_fft: int = 400#
property n_samples: int#

Samples in one encoder window (30 s at 16 kHz).

no_timestamps_token_id: int = 50364#
property num_frames: int#

Mel frames in one encoder window (3000); the encoder’s conv2 halves them to max_source_positions.

num_mel_bins: int = 128#
prev_sot_token_id: int = 50362#
prompt_tail_ids(task='transcribe', timestamps=False)[source]#

What follows the language token: <|task|> [<|notimestamps|>].

Parameters:
Return type:

list[int]

sampling_rate: int = 16000#
scale_embedding: bool = False#
suppress_tokens: list[int]#
task_to_id: dict[str, int]#
task_token(task)[source]#
Parameters:

task (str)

Return type:

int

property timestamp_begin: int#

<|0.00|>; token timestamp_begin + i means 0.02 * i s.

property timestamp_precision: float#
timestamp_seconds(token_id)[source]#
Parameters:

token_id (int)

Return type:

float

vocab_size: int = 51866#
Contents