EncDecCWE

class minicons.cwe.EncDecCWE(model_name: str, device: str | None = 'cpu', pretrained: bool = True)

Bases: CWE

Implements the contextualized word embedding class to facilitate extraction of word representations from an encoder-decoder transformer model.

encode_text(text: str | List[str], layer: int | List[int] = None) → Tuple

Encodes batch of raw sentences using the model to return hidden states at a given layer.

Parameters:

• text (Union[str, List[str]]) – batch of raw sentences

• layer (int) – layer from which the hidden states are extracted.

Returns:

Tuple (input_ids, hidden_states)