1 Repositories
Python lm_dataloader Libraries
Dataloader tools for language modelling
Installation: pip install lm_dataloader Design Philosophy A library to unify lm dataloading at large scale Simple interface, any tokenizer can be inte
5 Mar 25, 2022