# LLM Pruning and Distillation in Practice: The Minitron Approach

*Source* · `src-sreenivas-2024-minitron` · status: **n/a**

| field | value |
|---|---|
| kind | Preprint |
| authors | Sharath Turuvekere Sreenivas, Saurav Muralidharan, et al., Bryan Catanzaro |
| venue | arXiv |
| date | 2024-08-21 |
| url | https://arxiv.org/abs/2408.11796 |
| arxiv | 2408.11796 |
| accessed | 2026-10-24 |

## Notes

Depth and width pruning plus distillation: Llama 3.1 8B to 4B and Mistral NeMo 12B to Mistral-NeMo-Minitron-8B; slight teacher fine-tuning on the distillation set helps when original data is unavailable; weights released.

## Backlinks (computed)

**cited by:** [On-device small language models for characters](../technique/on-device-language-models.md)

## Citation

Sharath Turuvekere Sreenivas, Saurav Muralidharan, et al., Bryan Catanzaro (2024-08-21). *LLM Pruning and Distillation in Practice: The Minitron Approach*. arXiv https://arxiv.org/abs/2408.11796

---
Provenance: created in run `2026-10-24-techniques-llm-agents`.