1

The BERT-based SRL model that Shi and Lin develop (which is currently the the backend of the AllenNLP SRL model) has more consistent advantages over Ouichi et al.'s (2018) ensemble model when using BERT-large, instead of BERT-base. For example, the Shi and Lin model gets better F1 than Ouichi on CoNLL 05 only when using BERT-Large.

So, is it possible to use the AllenNLP SRL model with BERT-large rather than BERT-base?

7
  • 1
    This is definitely possible, you'd just need to fine-tune it using "bert-large". You could easily adapt this training config to do so: github.com/allenai/allennlp-models/blob/main/training_config/…
    – petew
    Aug 6, 2021 at 22:22
  • 1
    Yeup! Although you may need to adjust some hyperparameters like the learning rate.
    – petew
    Aug 9, 2021 at 15:46
  • 1
    Thanks! It seems to work (or at least, I get other errors seemingly unrelated to the base vs large, lol). As for the learning rate, the Shi and Lin paper seems to imply the same learning rate for base and large.... Aug 9, 2021 at 16:00
  • 1
    Yes you should be able to download those files locally. Then in your AllenNLP config, change the name of the pretrained model to the local path of your downloaded files (the directory).
    – petew
    Aug 23, 2021 at 15:46
  • 1
    Glad to hear it!
    – petew
    Aug 24, 2021 at 22:54

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.