ÚFAL at MultiLexNorm 2021:
Improving Multilingual Lexical Normalization by Fine-tuning ByT5
David Samuel & Milan Straka
Charles University
Faculty of Mathematics and Physics
Institute of Formal and Applied Linguistics
Paper (TODO)
Interactive demo on Google Colab
HuggingFace models (TODO)
This is the official repository for the winning entry to the W-NUT 2021: Multilingual Lexical Normalization (MultiLexNorm) shared task, which evaluates lexical-normalization systems on 12 social media datasets in 11 languages.
Our system is based on ByT5, which we first pre-train on synthetic data and then fine-tune on authentic normalization data. It achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. In addition to these source files, we also release the fine-tuned models on HuggingFace (TODO) and an interactive demo on Google Colab.
How to run
🐾
Clone repository and install the Python requirements
git clone https://github.com/ufal/multilexnorm2021.git
cd multilexnorm2021
pip3 install -r requirements.txt
🐾
Initialize
Run the inialization script to download the official MultiLexNorm data together with a dump of English Wikipedia. We recommend downloading Wikipidia dumps to get clean multi-lingual data, but other data sources should also work.
./initialize.sh
🐾
Train
To train a model for English lexical normalization, simply run the following script. Other configurations are located in the config folder.
python3 train.py --config config/en.yaml
Please cite the following publication
@inproceedings{wnut-ufal,
title= "{ÚFAL} at {MultiLexNorm} 2021: Improving Multilingual Lexical Normalization by Fine-tuning {ByT5}",
author = "Samuel, David and Straka, Milan",
booktitle = "Proceedings of the 7th Workshop on Noisy User-generated Text (W-NUT 2021)",
year = "2021",
publisher = "Association for Computational Linguistics",
address = "Punta Cana, Dominican Republic"
}
29 Aug 22, 2022
70 Oct 29, 2022
266 Nov 24, 2022
2 Oct 30, 2022
68 Dec 9, 2022
40 Dec 22, 2022
3 Mar 16, 2022
12 Apr 30, 2022
41 Dec 14, 2022
94 Dec 25, 2022
9 Dec 1, 2022
249 Jan 3, 2023
37 Oct 30, 2022
24 Sep 8, 2022
104 Nov 22, 2022
62 Dec 28, 2022
45 Aug 31, 2022
10 Jul 14, 2022
20 Dec 9, 2021