Transformer-based language models use convolutional layers to apply filters across an input to capture local patterns through filtered views.

Transformer-based language models can process only text data.