Dec 2024
SLM — Marathi Language Model
Pretrained an 84M-parameter GPT-2-inspired Marathi LM from scratch (6 layers, 6 heads, 384 dim). Built a custom 32K Marathi tokenizer and a 66K+ story dataset by translating TinyStories.
- PyTorch
- GPT-2
- Tokenizer
- Pretraining
