StrAE: Autoencoding for Pre-Trained Embeddings using Explicit Structure
Published in EMNLP Main, 2023
We look at the benefits of explicit structure as a way of modelling composition, and the effects this has on representation learning.
Download here
.png)
Publications
On the effect of curriculum learning with developmental data for grammar acquisition
Published in CoNLL Shared Task: BabyLM Challenge, 2023
We take a look at whether a developmentally plausible curriculum is beneficial for pre-training.
Download here